Wordgram

Wordgram is a term used in information science and computational linguistics to describe a representation of words based on sequences of characters, or n-grams, within those words. The idea treats a word as a collection of overlapping character grams, which can be encoded into a vector for machine learning tasks. The term is not universally standardized and may be used differently in different studies.

Construction of a wordgram typically involves selecting a range of n-gram lengths, such as three to five characters. All distinct grams are enumerated, and each word is represented by a vector whose components indicate the presence or frequency of each gram in the word. Vectors can be binary, frequency-based, or weighted using information retrieval schemes like TF-IDF. Because the set of possible grams can be large, dimensionality reduction, hashing tricks, or feature selection are often employed to keep models tractable.

Applications for wordgram representations include text classification, language identification, authorship attribution, and spelling or OCR error correction, particularly in scenarios with typographical variation or morphologically rich languages. Wordgrams can capture subword information that traditional word-level features miss, contributing to robustness against misspellings and creative spellings.

Relation to and limitations of the approach: Wordgram features overlap with broader n-gram and subword modeling techniques. They can offer interpretability and resilience to noise but may be high-dimensional and sparse, necessitating regularization or dimensionality reduction. They are generally used alongside, rather than as a replacement for, learned word embeddings and other rich representations.

See also: n-gram, character n-gram, subword model, word embedding, bag of n-grams.