Hazm

Hazm is a Python library for Persian natural language processing. It provides a set of tools and resources designed to process Persian text, supporting tasks common in NLP workflows such as text normalization, tokenization, stemming and lemmatization, part-of-speech tagging, and basic parsing capabilities. The library is widely used in academic research and practical applications dealing with Persian language data.

Normalization in Hazm standardizes Persian text by handling character forms, spacing, and diacritics, helping to reduce variability in input data. Tokenization, or word segmentation, splits Persian text into tokens while accounting for Persian orthography and affixes. Stemming and lemmatization offer morphological processing to reduce words to their base forms, aiding downstream analysis. POS tagging assigns grammatical categories to tokens, enabling syntactic and semantic processing. Hazm also provides utilities for sentence splitting and other preprocessing steps that commonly appear in Persian NLP pipelines.

Hazm is open-source and maintained on platforms like GitHub. It can be installed in Python environments via package managers, typically using commands such as pip install hazm. The library is designed to be accessible for researchers and developers working with Persian text and is compatible with Python versions widely used in data science workflows.

Limitations and scope reflect the state of Persian NLP tooling. Hazm performs well on standard Persian texts but may face challenges with dialects, highly informal writing, or domain-specific language. Like many open-source NLP tools, its accuracy depends on the data it was designed and tested with, and users may need to complement it with domain-specific resources for best results.

Hazm sits within the broader field of Persian NLP, alongside other libraries and resources that support language technologies for Persian.