liikemääräohjaimia
Liikemääräohjaimia, or momentum controllers, are a class of algorithms used in machine learning, particularly in training artificial neural networks. They aim to accelerate the convergence of the training process and help overcome issues like vanishing or exploding gradients. The core idea behind momentum is to introduce a fraction of the previous update vector into the current update. This helps the optimizer "roll down" the cost function landscape more smoothly, similar to a ball rolling down a hill.
The standard momentum method, often referred to as "heavy ball" momentum, adds a term proportional to the
v_t = gamma * v_{t-1} + eta * nabla_theta J(theta_t)
theta_{t+1} = theta_t - v_t
where v_t is the update vector at time t, gamma is the momentum coefficient (typically between 0
Another popular variant is Nesterov Accelerated Gradient (NAG). NAG modifies the standard momentum approach by calculating
Liikemääräohjaimia are effective in situations where the cost function has shallow gradients or is noisy. By