labelwhat

LabelWhat is an open-source, modular platform for data annotation designed to support labeling workflows across multiple data modalities used in machine learning, including images, text, and audio. It provides a centralized workspace for defining label taxonomies, coordinating annotator tasks, validating annotations, and exporting labeled datasets into common formats. The project aims to promote reproducibility and collaboration by preserving provenance, versioning label schemas, and recording adjudication decisions.

LabelWhat supports hierarchical and multi-label taxonomies, per-task assignment, review queues, and quality-control mechanisms such as adjudication and inter-annotator agreement metrics. It provides a plugin architecture and APIs to integrate with storage, preprocessing, and model-training pipelines. Exports include COCO, Pascal VOC, JSON Lines, and CSV, with configurable preprocessing for normalization. The front-end offers drawing tools for images, text spans, and audio transcripts, plus keyboard-driven workflows.

Architecture and workflow: LabelWhat follows a client-server model. A central server stores label schemas, task definitions, and annotation records in a structured database, while a web-based front-end lets annotators work asynchronously. A taxonomy graph underpins the labeling schema, enabling inheritance, synonyms, and validation rules. Workflows support task creation, assignment, progress tracking, and dispute resolution.

History and reception: The project originated from a collaboration among researchers and practitioners in 2020 and was released as an open-source project in 2021. Since then, it has attracted contributors and users in academia and industry, particularly for multimedia and NLP labeling projects. Critics note the learning curve and the need for careful schema design to avoid labeling drift.

See also: data annotation tools, active learning, annotation schema.