WLS

WLS, or Weighted Least Squares, is a statistical method that generalizes ordinary least squares (OLS) to handle heteroscedasticity or varying measurement precision across observations. Instead of minimizing the sum of squared residuals, WLS minimizes a weighted sum, giving more or less influence to each observation according to its reliability.

In the standard formulation, given a linear model y = Xβ + ε with E(ε) = 0 and Var(ε) = Σ, WLS minimizes the objective function sum w_i (y_i - x_i^T β)^2, where w_i > 0 are weights. In matrix form, this is minimized by the estimator β̂_WLS = (X^T W X)^{-1} X^T W y, with W being a diagonal matrix whose diagonal entries are the weights w_i. If all weights are equal, WLS reduces to ordinary least squares.

Weights are typically chosen to reflect the reliability or variance of each observation. A common choice is w_i ∝ 1 Var(y_i | x_i), so observations with larger variance receive less weight. If the form of Var(y_i) is known, W can be set accordingly; when variances are unknown, they may be estimated from data or treated iteratively.

WLS is a special case of generalized least squares (GLS) in which the error covariance is assumed diagonal (i.e., uncorrelated errors with unequal variances). If errors are heteroskedastic but uncorrelated, WLS is appropriate; if errors are also correlated, GLS with a non-diagonal covariance matrix is used.

Applications include econometrics, psychometrics, and survey analysis, where measurement precision or sampling weights vary across observations. Limitations arise if weights are misspecified, which can reduce efficiency and affect standard error estimates.