opendatatxt

OpenDataTXT is an open data initiative and standard intended to facilitate the sharing and reuse of text datasets. It provides a common data model for describing documents, annotations, metadata, licensing, and provenance, with the aim of improving interoperability across natural language processing, digital humanities, and language technology research. The project emphasizes reproducibility, data quality, and transparent provenance.

The core data model centers on documents and their associated annotations, metadata, and rights information. A document entry can include the text itself or a pointer to a repository location, along with language, domain, and creation date metadata. Annotations reference text spans and may be linked to ontologies or schemas. Metadata covers provenance, contributor identity, source attribution, and version history. A dataset manifest records licensing, access terms, and any processing or redaction performed. The standard supports multiple serializations, notably JSON-LD, RDF, and CSV-based representations, and accommodates integration with existing formats such as TEI for scholarly corpora.

Licensing and governance are designed to balance openness with clarity. OpenDataTXT encourages permissive licenses (e.g., CC0 or CC BY) while allowing restricted licenses, provided terms are explicit. Governance is community-driven, typically involving a public review process, contribution guidelines, and version control via a transparent repository and discussion venues.

Adoption and impact include use by libraries, archives, universities, and NLP research groups seeking interoperable datasets and tooling. The ecosystem offers validators and tooling for data quality, license compliance, and metadata normalization, aiming to lower barriers to data sharing while ensuring ethical and legal considerations are respected.