lassoregression

LASSO regression, short for Least Absolute Shrinkage and Selection Operator regression, is a regression method that combines estimation and variable selection. It introduces a penalty based on the absolute values of the coefficients, which encourages some coefficients to shrink to exactly zero, effectively removing corresponding predictors from the model.

Mathematically, LASSO minimizes the objective function (1/2n) ||y − Xβ||^2_2 + λ ∑|β_j|, where y is the response vector, X is the design matrix, β is the coefficient vector, n is the number of observations, and λ ≥ 0 is a tuning parameter that controls the strength of the penalty. Larger values of λ increase sparsity, leading to simpler models with fewer active predictors. The method is a form of linear regression with L1 regularization.

LASSO is contrasted with ridge regression, which uses an L2 penalty and tends to shrink coefficients but rarely to zero, offering limited variable selection. By enabling coefficients to become exactly zero, LASSO improves interpretability and can enhance predictive performance in high-dimensional settings where p (predictors) exceeds n or where many predictors are irrelevant. The elastic net combines L1 and L2 penalties to address some limitations of LASSO, particularly its behavior with correlated predictors.

Common practice includes standardizing features before applying LASSO, because the penalty is scale-dependent. Lambda is typically selected via cross-validation, information criteria, or other model selection criteria. LASSO is widely used in genomics, economics, and other fields with high-dimensional data, and is implemented in software such as glmnet and scikit-learn.