KarhunenLoève

Karhunen–Loève refers to a foundational result and its associated transform used to represent a stochastic process as an expansion in orthogonal basis functions derived from the process’s covariance. For a second-order stochastic process X(t) defined on an interval with zero mean, there exists an orthonormal set of eigenfunctions φ_n(t) and nonnegative eigenvalues λ_n such that X(t) can be written as X(t) = sum_{n=1}^∞ ξ_n φ_n(t), where ξ_n = ∫ X(s) φ_n(s) ds and E[ξ_m ξ_n] = λ_n δ_mn. The φ_n are eigenfunctions of the covariance operator C(s,t) = E[X(s)X(t)], and the coefficients ξ_n are uncorrelated with zero mean and variances λ_n. The expansion is called the Karhunen–Loève expansion and is optimal in the sense that it minimizes mean-squared reconstruction error for a given number of terms.

The theorem was developed independently by Kari Karhunen (1947) and Ole Loève (1949) and is widely used as a continuous analogue of principal component analysis (PCA). In practice, a discrete dataset yields a finite covariance matrix, whose eigenvectors provide the principal directions and whose eigenvalues quantify explained variance. Truncated expansions form dimensionality reductions.

Applications span signal processing, data compression, noise reduction, spectral analysis, and functional data analysis. The Karhunen–Loève transform is particularly valuable for obtaining energy-efficient representations of stochastic signals with correlations captured by the covariance structure.