Unigram

Unigram refers to a single unit, typically a word, used in linguistic analysis and natural language processing. In the context of language modeling, a unigram model is an order-1 model that assumes each word occurs independently of the others. Under this assumption, the probability of a sentence w1 w2 ... wn is approximated as the product of the probabilities of its words: P(w1, w2, ..., wn) ≈ ∏i P(wi).

Computation of unigram probabilities is usually based on a text corpus. If N is the total number of tokens and count(w) is the number of times word w appears, then P(w) = count(w) / N. The vocabulary size V includes all observed words. Because real text contains words unseen in a corpus, smoothing methods such as Laplace smoothing add a small constant to counts to assign nonzero probability to unseen words: P̃(w) = (count(w) + α) / (N + αV).

Applications and limitations: Unigram models are simple and fast, and they serve as baselines for tasks such as text classification and information retrieval, or as components of more complex models. They ignore word order and context, limiting their ability to capture syntax or semantics. To model dependencies, higher-order n-grams (e.g., bigrams, trigrams) or neural language models are used. Unigram features, whether word-based or character-based, are also common in bag-of-words representations for text classification.