Korpuszok

Korpuszok, in Hungarian referred to as korpuszok (plural of korpusz), are large, structured collections of natural language data used for linguistic analysis and natural language processing. A korpusz is a machine-readable assembly of texts and related language data, often accompanied by metadata such as author, genre, date, and annotations when available. Korpuszok can be monolingual or multilingual, written or spoken, and vary widely in size. They may be raw text or include layers of annotation such as part-of-speech tagging, lemmatization, syntactic structure, discourse markers, or semantic tagging.

Korpuszok serve multiple purposes. They support studies of word frequency, collocations, grammar patterns, semantic fields, and language change over time. They underpin lexicography, language education, and many natural language processing tasks, including language modeling, tagging, parsing, and machine translation. Parallel corpora contain aligned translations and are particularly valuable for training and evaluating translation systems, while comparable corpora enable cross-language studies without exact alignments.

Creating and maintaining korpuszok involves data collection, cleaning, normalization, and annotation, along with quality control. Important considerations include copyright and licensing, representativeness, sampling bias, and the coverage of genres and registers. Access to korpuszok ranges from publicly available online resources to restricted institutional databases, often with specific usage terms and citation requirements.