pLSA

Probabilistic Latent Semantic Analysis (PLSA) is a statistical method for uncovering latent topics in text corpora. It was introduced by Thomas Hofmann in 1999 as a probabilistic alternative to traditional factorization approaches. In PLSA, each document is assumed to be a mixture of latent topics, and each topic is a distribution over words. The core idea is that the joint probability of a document and a word can be expressed as a sum over topics of the probability of the topic given the document multiplied by the probability of the word given the topic.

Training PLSA typically uses the Expectation-Maximization algorithm to estimate the model parameters from observed document-word counts. In the E-step, the algorithm computes the posterior probability that a given word in a given document was generated from each topic. In the M-step, it updates the topic distributions over words and the document-specific mixtures of topics to maximize the likelihood of the observed data.

A notable limitation of PLSA is its tendency to overfit and its lack of a principled prior over per-document topic mixtures, which makes it less robust for handling new, unseen documents. Consequently, PLSA is often viewed as a precursor to Latent Dirichlet Allocation (LDA), which introduces Dirichlet priors over document-topic distributions to improve generalization and provide a fully generative model for new documents.

PLSA has been applied to tasks such as topic modeling, information retrieval, and document clustering. Variants and extensions include approaches that adapt PLSA ideas to supervised settings or to alternative data modalities, sometimes under the umbrella term probabilistic latent semantic indexing.