ngrammen

Ngrammen (n-grams) are contiguous sequences of n items drawn from a text or speech stream. The items can be characters, syllables, or words. For example, in the sentence "the quick brown fox," unigrams are the single words, bigrams are “the quick,” “quick brown,” and “brown fox,” while trigrams are “the quick brown” and “quick brown fox.” The distinction between word-level and character-level n-grams is common: word n-grams emphasize lexical content, while character n-grams capture orthographic and subword patterns.

In language modeling, n-gram models estimate the probability of a token given the preceding n−1 tokens. This is typically written as P(wi | wi−n+1 … wi−1), based on a sliding window over the text. Models are built by counting n-gram frequencies from a corpus and applying smoothing to handle unseen sequences. Applications include predictive text, speech recognition, machine translation, information retrieval, and text classification. N-grams are also used in authorship attribution and plagiarism detection, as they can reflect stylistic or topical patterns.

Limitations of n-gram approaches include data sparsity as n grows, leading to high-dimensional representations and reliance on large corpora. They also assume a Markov property, limiting long-range dependencies. Tokenization and language morphology affect results, and higher-order models may overfit. Practical use often combines word and character n-grams or uses smoothing and dimensionality reduction. The concept has historical roots in information theory and has been widely adopted in natural language processing since the late 20th century, remaining a foundational tool for textual analysis.