SteepestDescentMethode

SteepestDescentMethode, commonly known as the steepest descent or gradient descent method, is a classical algorithm for unconstrained optimization of differentiable functions. The central idea is to move at each iteration in the direction of steepest decline of the objective function, which under the Euclidean norm is the negative gradient.

Given a differentiable function f: R^n → R and a current point x_k, the descent direction is p_k = −∇f(x_k). A step is taken along this direction to obtain x_{k+1} = x_k + α_k p_k, where α_k ≥ 0 is a step size chosen to reduce f as much as possible along the line x_k + α p_k. In exact line search, α_k minimizes f(x_k + α p_k) with respect to α; in practice, inexact line search or a fixed step size is often used.

For quadratic functions with a symmetric positive definite Hessian, a closed-form expression can determine the optimal step length: α_k = (∇f_k^T ∇f_k) / (∇f_k^T A ∇f_k). In general nonlinear problems, convergence to a local minimum is achieved under mild conditions (e.g., convexity or Lipschitz continuity of the gradient), but the method can be slow, especially on ill-conditioned problems where the path zigzags along narrow valleys.

Variants and enhancements include line search strategies, fixed or adaptive step sizes, and preconditioning to improve convergence. The steepest descent method serves as a simple baseline and is related to more advanced techniques such as conjugate gradient, which aim to remedy slow convergence in many practical cases. It remains a foundational concept in optimization theory and practice.