corpusarbete

Corpusarbete refers to the systematic development and use of language corpora for linguistic research, language technology and lexicography. It involves designing corpora, collecting data, annotating it with linguistic layers, and making the resources accessible to researchers and developers. The goal is to provide empirical, reproducible data about how language is used in real contexts.

Data collection may include written texts and spoken recordings from diverse sources. Licenses, privacy considerations and copyright restrictions shape what may be included. Metadata describes the corpus in terms of domain, genre, time period and linguistic features. Annotation can cover parts of speech, lemmas, morphological tags, syntactic structures, semantic roles and discourse features. Guidelines and inter-annotator agreement are important for quality control.

Processing workflows typically involve preprocessing (cleaning, normalization and tokenization), annotation, validation and versioning. Researchers use concordancers, frequency lists and collocation analyses, and may build or train language technology models on the corpus. Parallel corpora require alignment between languages.

In Sweden and other European contexts, corpusarbete is supported by national and international infrastructures such as Språkbanken and the CLARIN network, which provide access to corpora, tools and metadata standards. Data formats often rely on TEI, XML or JSON.

Applications include linguistic research, lexicography, education and NLP tool development. Challenges include ensuring representativeness, dealing with bias, managing copyright and privacy, and maintaining annotation consistency across projects and over time.