LSA

Latent Semantic Analysis (LSA) is a collection of techniques in natural language processing and information retrieval that analyzes relationships between a set of documents and the terms they contain by discovering latent semantic structures. It aims to capture associations between terms that co-occur across documents, enabling the inference of meaning and semantic similarity even when exact keywords differ.

LSA constructs a term-document matrix, with terms as rows and documents as columns. Entries reflect term frequency and may be weighted using approaches such as tf-idf. The matrix is then reduced in dimensionality through singular value decomposition (SVD), producing a low-rank representation in which each dimension corresponds to latent concepts or topics. By projecting documents and terms into this latent semantic space, LSA supports tasks such as information retrieval, document clustering, and measurement of semantic similarity via cosine distance.

LSA was introduced in its standard form by Deerwester, Dumais, Furnas, Landauer, and Harshman in 1990. It became influential in information retrieval and cognitive modeling by providing a mathematically principled way to address synonymy and polysemy. In practice, LSA can improve search results by linking queries to semantically related terms that do not share the exact keywords.

Limitations include the need for large, representative corpora, sensitivity to the chosen dimensionality, and challenges in interpreting the latent dimensions. As a linear algebra-based method, LSA may not capture complex, non-linear relationships as effectively as modern neural embeddings. Today, word and document embeddings from models such as word2vec, GloVe, and transformers have largely supplanted LSA in many applications, though LSA remains a useful baseline, teaching tool, and lightweight option for certain tasks.