L1regularized

L1 regularized refers to a regularization technique in statistical modeling and machine learning that adds a penalty proportional to the sum of the absolute values of the model coefficients. The goal is to prevent overfitting and improve generalization by constraining the size of the coefficients. The strength of the penalty is controlled by a hyperparameter, often denoted lambda, which trades off fit to the data against coefficient magnitude.

In a typical setting, such as linear or generalized linear models, the objective becomes the loss function plus lambda times the L1 norm of the coefficient vector. For example in linear regression this takes the form: minimize sum of squared errors plus lambda times the sum of absolute coefficients. The L1 penalty is non-differentiable at zero, which is central to its tendency to drive some coefficients exactly to zero. This produces sparse solutions that can perform feature selection implicitly.

Optimization with L1 regularization commonly uses subgradient methods, coordinate descent, or proximal gradient techniques with soft-thresholding. Elastic net, which combines L1 and L2 penalties, is often used to address potential instability of pure L1 regularization in highly correlated feature sets.

Compared with L2 (ridge) regularization, L1 tends to produce sparse models, which can enhance interpretability and reduce variance, but may introduce bias and can be less stable when features are highly correlated. L1 regularization is widely used in sparse learning, compressed sensing, and high-dimensional problems, where feature selection is desirable and the number of predictors is large relative to observations.