HomotopyLARS

HomotopyLARS is a computational approach that blends homotopy continuation with the Least Angle Regression (LARS) framework to compute the regularization path of L1-penalized linear models, notably the LASSO. The method treats the solution of the L1-regularized problem as a continuous path in the regularization parameter, tracing β(λ) from a simple starting point (often β = 0 at large λ) to the target model as λ decreases.

The algorithm relies on the observation that, under certain conditions, the L1-regularized solution changes in a piecewise-linear fashion as λ varies. The homotopy path is constructed by maintaining an active set A of nonzero coefficients and solving for the coefficient values along a direction that preserves optimality. When a breakpoint is reached, either a new variable enters the active set or an existing one leaves, and the tracking updates accordingly.

In practice, HomotopyLARS proceeds by calculating breakpoints as the minimum over candidate events (e.g., a residual correlation reaching λ for an inactive variable or a coefficient hitting zero). Between breakpoints, β_A moves linearly with λ, and the residual is adjusted in parallel.

Connections: The approach generalizes LARS for the LASSO path and intersects with standard homotopy continuation methods used in sparse recovery. It is particularly suitable for high-dimensional settings where p exceeds n, enabling efficient computation of the full regularization path without re-solving from scratch at every λ. Applications include sparse regression, feature selection, and model selection in statistics and machine learning.

See also LARS, LASSO, homotopy continuation, sparse regression.