L1Lasso

L1Lasso, commonly referred to simply as Lasso, is a regularization technique used in linear modeling to improve prediction accuracy and perform feature selection. It applies an L1 penalty to the model coefficients, encouraging sparsity by shrinking some coefficients exactly to zero. This makes L1Lasso particularly useful in high-dimensional settings where the number of predictors can be large relative to the number of observations.

Mathematically, L1Lasso solves the optimization problem that minimizes the residual sum of squares with an L1 penalty on the coefficients. The standard formulation is: minimize (1/2n) ||y - Xβ||^2_2 + λ||β||_1, where y is the response vector, X the design matrix, β the coefficient vector, and λ ≥ 0 the regularization parameter. The parameter λ controls the trade-off between fitting the data and enforcing sparsity: larger values lead to more coefficients being set to zero. The approach extends to generalized linear models, such as logistic regression, by replacing the least-squares loss with the appropriate loss function while keeping the L1 penalty on the coefficients.

Computation for L1Lasso is typically carried out with algorithms such as coordinate descent or proximal gradient methods, which efficiently handle the nondifferentiable L1 term. In practice, features are often standardized before fitting to ensure comparability of penalties across predictors. The L1 penalty not only shrinks coefficients but also performs variable selection, aiding interpretability.

Choosing the regularization strength is commonly done via cross-validation or information criteria. L1Lasso is related to, but distinct from, Elastic Net, which combines L1 and L2 penalties to address issues with correlated predictors. Historically, Lasso was introduced by Tibshirani in 1996 as a method for simultaneous estimation and variable selection.