Subwords

Subwords are linguistic or computational units that are smaller than a word. In general, the term refers to any string of characters that forms part of a word, or to the meaningful morphemes that constitute words. Subword analysis is used to study how words are built from smaller parts, as well as to model language in ways that can generalize across related words.

In linguistics, subwords include morphemes and affixes and other meaningful subunits. A bound morpheme such as prefix un- or suffix -ness cannot stand alone but contributes to meaning. A word like happiness can be analyzed as the sequence un- + happy + -ness, illustrating how subparts carry semantic weight and grammatical function. Not all subparts are morphemically meaningful; in some analyses, syllables or other segments are treated as subwords for phonological or psycholinguistic reasons.

In natural language processing, subword tokenization reduces vocabulary size and handles out-of-vocabulary words. Algorithms such as Byte-Pair Encoding (BPE), WordPiece, and SentencePiece learn a fixed set of subword units from a corpus. New words are then split into sequences of these units, enabling the model to process unseen words by composition. This approach balances representational efficiency with coverage, and can share subword units across languages in multilingual models.

Subword methods are especially beneficial for morphologically rich languages and languages with productive compounding, where a traditional word-level vocabulary would be impractically large. They raise considerations about granularity (how long the units should be) and linguistic alignment (whether subwords align with semantic or morphological boundaries). Tradeoffs include slightly increased decoding or tokenization complexity and potential ambiguity in unit boundaries.

Subwords thus occupy a central role in linguistics and computational linguistics, bridging the study of internal word structure with practical modeling of language data.