randomsearch

Random search is a straightforward stochastic optimization method that explores a search space by generating candidate solutions at random and evaluating them with an objective function. Candidates are drawn from a predefined probability distribution over the feasible region, commonly a uniform distribution. The approach is derivative-free, robust to non-smooth or noisy objectives, and does not require gradient information.

The typical workflow defines the search space and a budget of evaluations. At each iteration, a candidate solution is sampled according to the chosen distribution, the objective function is evaluated, and the best candidate seen so far is recorded. After the allotted budget is exhausted, the method returns the best found solution. Random search can operate in continuous, discrete, or mixed spaces, and the sampling strategy can be uniform or informed by prior knowledge or empirical results.

Applications of random search include hyperparameter optimization in machine learning, where it can tune settings such as learning rates, regularization strengths, and architectural choices. It is also used in engineering design, simulation-based optimization, and other domains where objective landscapes are complex, expensive to evaluate, or lack gradient information.

Advantages of random search include its simplicity, easy parallelization, and applicability to a wide range of problems without derivative information. Its main limitation is inefficiency in high-dimensional spaces or when the objective has a small number of influential factors, since many evaluations may yield little improvement. Variants and improvements include adaptive or guided random search, where sampling distributions evolve over time, and methods that combine random search with local search, Bayesian ideas, or quasi-random sequences to improve efficiency.