ngramit

ngramit is a Python library designed for the efficient generation and manipulation of n-grams from text. An n-gram is a contiguous sequence of n items from a given sample of text or speech. For example, in the sentence "the quick brown fox", the 2-grams (or bigrams) are "the quick", "quick brown", and "brown fox".

The ngramit library provides tools to extract n-grams of various sizes (unigrams, bigrams, trigrams, etc.) from a given string or list of strings. It supports customizable tokenization, allowing users to define how words or characters are separated. The library also offers functionalities for counting n-gram occurrences and for filtering n-grams based on certain criteria.

N-gram analysis is a fundamental technique in natural language processing (NLP) and is widely used in applications such as text classification, machine translation, speech recognition, and spell checking. By analyzing sequences of words or characters, it's possible to capture patterns and dependencies within language that individual words alone might miss. ngramit aims to simplify the process of generating and working with these sequences for researchers and developers. The library is implemented in Python, making it accessible to a large community of developers familiar with the language. Its focus is on providing a straightforward and performant solution for n-gram extraction tasks.