RidgeLasso

Ridgelasso is a term used in statistics and machine learning to describe a regularization technique that blends ridge (L2) and lasso (L1) penalties in linear or generalized linear models. The aim is to combine the stability and shrinkage of ridge regression with the sparsity-promoting properties of the L1 penalty. The name is informal and not standardized in the literature; the approach is more commonly referred to as the elastic net.

A common mathematical formulation minimizes the residual sum of squares with two regularization terms: minimize (1/2n) ||y - Xβ||^2 + λ1 ||β||_1 + λ2 ||β||_2^2, where λ1 and λ2 control the strength of the L1 and L2 penalties, respectively. An equivalent and widely used parameterization introduces a mixing parameter α in [0,1] and a single tuning λ: minimize (1/2n) ||y - Xβ||^2 + λ [ (1 - α)/2 ||β||^2_2 + α ||β||_1 ]. This form highlights the balance between sparsity and stability.

Optimization for ridgelasso-type models is typically performed via coordinate descent or proximal gradient methods, with feature standardization often recommended to ensure penalties affect coefficients comparably.

Applications and interpretation: ridgelasso is suited for regression problems with many predictors, especially when predictors are correlated and some degree of variable selection is desired. It tends to select groups of correlated variables rather than a single representative, and its performance depends on the choice of regularization parameters.

Relation to Elastic Net: while ridgelasso conveys the same principle, elastic net is the established, widely used name for this combined regularization approach. Ridgelasso may appear in informal discussions or specific implementations. See also elastic net, ridge regression, lasso.