LDA

Latent Dirichlet Allocation (LDA) is a generative probabilistic model for collections of discrete data such as text corpora. It represents each document as a mixture of latent topics, where each topic is a probability distribution over words.

The model assumes the following generative process: for each document, sample a topic distribution from a Dirichlet prior; for each word, sample a topic from the document-specific distribution; then sample the word from the topic's word distribution. The Dirichlet priors with parameters alpha and beta control sparsity and smoothness of the topic and word distributions.

Inference aims to recover the hidden topic structure given observed documents. Exact inference is intractable; approximate methods include variational Bayes and Gibbs sampling, with online or streaming variants for large corpora.

Applications of LDA include discovering themes in large text collections, document clustering, topic-based document retrieval, summarization, and trend analysis. Its strengths include being unsupervised, scalable to large datasets, and yielding interpretable topics. Limitations include the bag-of-words assumption that discards word order and syntax, potential difficulty in interpreting some topics, the need to select the number of topics K, and sensitivity to hyperparameters and initialization.

History and naming: LDA was introduced by David Blei, Andrew Ng, and Michael Jordan in 2003. It has since become a foundational method in topic modeling. The acronym LDA may also refer to Linear Discriminant Analysis in statistics, a supervised dimensionality reduction technique unrelated to topic modeling.