lowrank

Low-rank refers to a matrix whose rank is small relative to its dimensions. The rank is the maximum number of linearly independent rows or columns, equivalently the dimension of the column space. A low-rank matrix has most of its information contained in a small number of factors, which enables compact representation and efficient computation.

In data analysis, many matrices arising from observations are approximately low-rank, meaning they can be well approximated by a product UV^T with a small inner dimension. This property allows procedures to capture dominant structure while reducing noise and redundancy, aiding tasks such as compression and denoising.

The most common method for obtaining a low-rank approximation is the singular value decomposition (SVD). The Eckart-Young theorem states that the best rank-k approximation to a given matrix in Frobenius norm is given by the sum of the top k singular values and their corresponding singular vectors. If the exact rank-k structure is unknown, truncated SVD is used.

Optimization approaches use the nuclear norm, the sum of singular values, as a convex surrogate for rank. Nuclear-norm minimization underpins problems like matrix completion (recovering missing entries) and robust principal component analysis (separating low-rank structure from sparse errors).

Low-rank matrix factorization expresses a matrix as UV^T with U and V of smaller inner dimension, enabling scalable algorithms and applications in collaborative filtering, recommender systems, and dimensionality reduction. Other methods include probabilistic and Bayesian formulations.

Limitations include identifiability issues and the need for assumptions such as incoherence and sufficient observations for reliable recovery. In practice, low-rank models balance simplicity and fidelity, capturing core structure while ignoring noise.