matrixfactorization

Matrix factorization refers to a family of techniques for decomposing a matrix into the product of two or more smaller matrices. The canonical form seeks a low-rank approximation M ≈ U V^T, where M is an m×n matrix and U is m×k, V is n×k with k < min(m,n). This yields latent factors that can reveal structure in the data. Classic examples include singular value decomposition (SVD), which decomposes M into U Σ V^T with orthogonality constraints; and non-negative matrix factorization (NMF), which adds non-negativity constraints to produce interpretable factors.

Probabilistic and optimization-based variants, such as probabilistic matrix factorization (PMF), Bayesian PMF, and collaborative filtering approaches, model observed entries of M with latent user and item vectors and optimize a likelihood-based objective, often with regularization. In recommender systems, the matrix consists of user ratings or interactions, and factorization produces user and item latent factors that can predict missing ratings by dot product. Algorithms include alternating least squares (ALS), stochastic gradient descent (SGD), and coordinate descent; these handle missing data by optimizing only observed entries or by incorporating a probabilistic framework for missingness.

Applications span recommender systems, data compression, topic modeling, and computer vision. Limitations include the need to choose a rank k, potential non-convex optimization leading to local minima, cold-start problems for new users or items, and sensitivity to regularization choices. Extensions include integrating side information, sparsity constraints, and non-linear factorization methods.