stemmetesting

Stemmetesting is a methodological approach in linguistics and natural language processing used to evaluate the validity and productivity of proposed word stems across a language's morphology. It focuses on whether a given stem can support the expected inflectional and derivational patterns and remain stable under morphophonemic alternations. The term is a compound of stem and testing, emphasizing the empirical verification of stem-formhood rather than relying solely on lexeme listings.

Methodology: starting with a set of candidate stems, typically derived from lemmatization, a researcher applies a range of affixes and compounding environments. For each stem, the generated forms are matched against observed data in corpora or lexicons. Metrics such as coverage, precision, recall, and a stem-stability score are used to judge productivity. The process may use finite-state transducers, morphophonemic rules, or statistical language models to simulate inflected forms.

Applications: stemmetesting informs the development of stemming algorithms, morphological analyzers, and lexicon curation. It supports cross-linguistic comparison of morphological productivity and can help identify irregular or suppletive patterns that require separate treatment.

Limitations and challenges: the method depends on data quality and size; irregular morphologies, allomorphy, and orthographic variation can complicate stem validation. Distinguishing productive stems from borrowed or loanword stems also poses difficulties.

See also: stems, stemming, lemmatization, morphology, morphological productivity, computational linguistics.