OAEI

OAEI stands for the Ontology Alignment Evaluation Initiative. It is a community-driven, annual benchmarking campaign designed to evaluate ontology alignment (or ontology matching) systems and methods. The initiative provides shared datasets, reference alignments, and evaluation protocols to enable fair and reproducible comparison of different approaches.

Tracks and data sets in OAEI cover a range of alignment tasks and domains. Each track offers pairs of ontologies and corresponding gold standard mappings, allowing submitted systems to produce alignments that are then measured against these reference mappings. Tracks may focus on general-purpose ontology matching, domain-specific challenges such as biomedicine, or cross-domain and multilingual alignment, among other variations.

Evaluation and scoring typically rely on standard information retrieval metrics such as precision, recall, and F-measure, to quantify the accuracy and completeness of the produced mappings. Some tracks may incorporate additional metrics or qualitative analyses to assess aspects like scalability, coherence, or runtime performance.

Results are published in annual evaluation reports, which present per-system scores, rankings, and analyses. These reports help identify strengths and limitations of different approaches, foster transparency, and provide benchmarks to guide future research and development in ontology alignment.

History and governance: OAEI emerged from the Semantic Web and knowledge engineering communities as a collaborative effort to standardize benchmarking for ontology alignment. It is coordinated by researchers and institutions affiliated with the broader community and is widely cited as a central resource for assessing alignment technologies. Access to datasets, tracks, and results is public, supporting ongoing research and comparison across years.