variancecovariance

The variance-covariance matrix, commonly called the covariance matrix, is a key object in multivariate statistics that summarizes how a set of random variables vary together. For a k-dimensional random vector X = (X1, ..., Xk) with mean μ, the covariance matrix is Σ = Cov(X) with elements Σ_ij = Cov(X_i, X_j) = E[(X_i − μ_i)(X_j − μ_j)]. The diagonal entries Σ_ii are the variances Var(X_i).

Properties of Σ include that it is symmetric and positive semidefinite. If the variables are independent, the off-diagonal elements are zero, and Σ is diagonal. When the variables have a joint normal distribution, Σ, together with μ, determines the distribution of X.

Transformations and estimation: For a linear transformation Y = AX + b, Cov(Y) = A Σ A^T. The sample covariance matrix S, computed from n observations x_i, is S = (1/(n−1)) ∑_i (x_i − x̄)(x_i − x̄)^T. Σ is often estimated from data by S, especially in multivariate analyses.

Relation to correlation: The correlation matrix R is obtained by standardizing Σ: R_ij = Σ_ij / sqrt(Σ_ii Σ_jj). R is dimensionless and has ones on the diagonal, reflecting unit correlation with itself.

Applications: The variance-covariance matrix underpins multivariate normal modeling, portfolio optimization in finance, principal component analysis, and many estimation procedures. It provides a concise description of the dependence structure among multiple variables.