underregularization

Underregularization is a term used in statistics and machine learning to describe a situation in which a model is not sufficiently constrained by regularization. When the penalty for complexity is too small or absent, the model can fit noise in the training data, leading to high variance and poor generalization. It is the opposite of overregularization; while overregularization can cause underfitting, underregularization tends to overfit.

Common causes include choosing a very small regularization strength (for example, a low lambda in ridge or lasso, or no weight decay in neural networks), using highly flexible models, and inadequate cross-validation that fails to reveal the need for stronger constraints.

Consequences include low training error but high validation or test error, indicating overfitting. Coefficients or weights may reach large magnitudes, and predictions can be sensitive to small input changes or to the particular training set.

Detection methods include comparing learning curves for training and validation sets, monitoring the gap between train and test performance, and inspecting parameter magnitudes. Cross-validation can highlight the need for stronger regularization.

Mitigation strategies involve increasing the regularization strength or selecting a more appropriate penalty (L1, L2, elastic net), reducing model complexity, and using techniques such as weight decay, dropout, or early stopping in neural networks. Data preprocessing and feature selection can also help reduce the risk of underregularization.