residuals

Residuals are the differences between observed outcomes and their predicted values from a statistical model, typically e_i = y_i − ŷ_i. In regression analysis they measure the discrepancy between data and the fitted model and are used to assess model fit. If the model is correctly specified with an intercept, residuals have expected mean zero; their variance and independence are assumptions that underpin inference. Large residuals may indicate outliers or mis-specification.

Common residual types include raw residuals, standardized residuals (dividing by an estimate of the residual standard deviation), and studentized residuals (which account for the influence of a point and its leverage). For generalized linear models, deviance residuals and Pearson residuals are used, while the leverage h_ii from the hat matrix identifies observations with unusual influence. These measures aid diagnostic checks and robustness assessments.

Diagnostics rely on residuals to detect nonlinearity, heteroscedasticity, autocorrelation, or outliers. Residual plots against fitted values or against time order, along with normal QQ plots of residuals, are standard tools. Influential observations are gauged with statistics such as Cook's distance and DFFITS, which quantify how much a point would alter the fitted model if removed.

In time-series analysis, residuals are the differences between observed values and model forecasts after fitting an autoregressive or other dynamic model. They should resemble white noise if the model captures the systematic structure; patterns or autocorrelation in residuals suggest model misspecification or remaining structure to be modeled.