Autoregressions

Autoregressions are a class of statistical models for time series in which the current observation is expressed as a function of its own past values and a stochastic term. The most common form is the autoregressive model of order p, denoted AR(p). It specifies X_t = c + phi1 X_{t-1} + ... + phip X_{t-p} + epsilon_t, where epsilon_t is white noise and c is a constant. The model assumes stationarity, meaning its statistical properties do not change over time; this requires the roots of the characteristic polynomial 1 - phi1 z - ... - phip z^p = 0 to lie outside the unit circle.

Estimation and model selection: Parameters phi1 through phip can be estimated by ordinary least squares for a fixed p, by Yule-Walker equations, or by maximum likelihood methods. The order p is typically chosen using information criteria such as AIC or BIC, or via cross-validation adapted for time-series. Diagnostics involve examining residuals for randomness and absence of autocorrelation.

Relation to other models: Autoregressions are central to broader families such as ARIMA, which incorporate differencing (I) and moving-average terms (MA). They generalize to multivariate settings in vector autoregression (VAR). Compared with moving-average models, autoregressive models express the current value as a function of past observations rather than past errors.

Applications and limitations: Autoregressions are widely used for forecasting in economics, finance, meteorology, and other domains with temporal structure. They require stationary data or appropriate transformations (e.g., differencing). Limitations include sensitivity to structural breaks, potential model misspecification from incorrect order, and assumptions of linearity and homoscedasticity. In machine learning, autoregressive ideas extend to sequential data generation by modeling the conditional distribution P(X_t | X_{<t}).