Derivaatittomat

Derivaatittomat is a class of optimization algorithms that solve problems without using derivatives. They rely on evaluating the objective function at selected points and using those results to guide the search for minima or maxima. This makes them suitable for black-box problems where gradients are unavailable, unreliable, or expensive to obtain.

The concept aligns with derivative-free optimization, a field whose roots lie in early direct-search methods such as Hooke–Jeeves and the Nelder–Mead simplex algorithm from the 1960s. Over time, more systematic approaches have been developed, including Mesh Adaptive Direct Search (MADS) and various evolutionary strategies like CMA-ES, which do not rely on gradient information and can handle nonlinear or noisy landscapes.

Derivaatittomat methods encompass several families. Direct search methods, such as pattern search and the Nelder–Mead algorithm, explore the search space using function evaluations arranged in simple geometric patterns. Surrogate-based methods build inexpensive models (for example, Gaussian processes) to approximate the objective and guide evaluations, as in Bayesian optimization. Evolutionary and population-based strategies explore many candidate solutions simultaneously and adapt their search based on observed performance, often performing well in higher dimensions.

Applications of Derivaatittomat techniques are widespread in engineering design, physics-based simulations, and machine learning, particularly for hyperparameter tuning or when objective evaluations are costly, noisy, or non-differentiable. They are also used in economics, operations research, and any domain where derivative information is unavailable or impractical to obtain.

Strengths include robustness to non-smooth or noisy objectives and independence from gradient calculations. Limitations can include slow convergence, high demand for function evaluations, and sensitivity to algorithmic parameters, which may require careful tuning for each problem.