overfit

Overfitting, commonly referred to as overfit in informal usage, is a modeling problem in statistics and machine learning where a model learns patterns that are specific to the training data rather than the underlying relationship. An overfit model may achieve near-perfect accuracy on the training set but perform poorly on new, unseen data because it has captured noise, outliers, or incidental idiosyncrasies.

Causes include excessive model complexity relative to the amount and quality of available data, insufficient data, features that are highly correlated with the target only in the training sample, and training procedures that allow the model to adapt too closely to the training data. Data leakage and improper validation can also create an illusion of low error during training.

Indicators include a large gap between training performance and validation or test performance, high variance across different data splits, and decreasing generalization as model complexity grows. A model that fits the training data well but generalizes poorly is typically overfitting.

Prevention and remedies: use proper data splitting and cross-validation; simplify the model by reducing parameters or features; apply regularization methods such as L1 or L2 penalties; use early stopping during iterative training; employ dropout or other regularizers in neural networks; augment data or collect more data; and consider ensemble methods or model selection guided by validation performance. The goal is to balance bias and variance to improve performance on unseen data.