PredictionError

Prediction error is the discrepancy between an observed outcome and the value predicted by a model or forecast. If Y denotes the true outcome and Ŷ denotes the predicted value computed from input features X, the pointwise prediction error is e = Y − Ŷ. On a dataset, common summaries are mean squared error (MSE) and mean absolute error (MAE). The term is used across disciplines, including statistics, machine learning, econometrics, and forecasting.

Prediction error can refer to residuals observed on a training sample or to the generalization error—the expected prediction error on new data drawn from the same population. The latter is affected by data noise, model bias, and variance, and it is typically estimated via cross-validation or held-out test sets. The irreducible error, often called σ^2, represents variability in Y that cannot be explained by X.

In squared-error contexts, the bias-variance decomposition states that the expected squared prediction error is E[(Y − Ŷ)^2] = Bias(Ŷ)^2 + Variance(Ŷ) + σ^2. This decomposition clarifies how model complexity trades off bias against variance, influencing overall predictive performance.

Common approaches to reduce prediction error include choosing appropriate models, regularization, feature engineering, and larger or higher-quality datasets. Evaluation uses metrics such as MSE, RMSE, MAE, and MAPE, often with cross-validation to approximate performance on unseen data. In neuroscience and reinforcement learning, a related concept is the reward prediction error, the difference between received and predicted rewards, guiding learning signals in the brain.