XNLI

XNLI, or Cross-lingual Natural Language Inference, is a benchmark dataset designed to evaluate cross-lingual natural language understanding. It was released by Facebook AI Research in 2018 as an extension of the English-language Multi-Genre Natural Language Inference (MNLI) corpus, expanded to cover a broad set of languages to test cross-language transfer in natural language inference tasks.

The core task in XNLI is natural language inference. For each example, a model is given a premise and a hypothesis and must assign one of three labels: entailment, contradiction, or neutral. In XNLI, the training data come from the English MNLI training set, while the development and test sets are translated into fourteen other languages, for a total of fifteen languages in the benchmark. The primary evaluation metric is accuracy on the target-language test sets.

Languages and scope: XNLI provides multilingual data across diverse languages and scripts, enabling research into cross-lingual transfer with both monolingual and multilingual models. The design supports evaluating how well a model trained in English, or trained on multilingual data, generalizes to other languages.

Impact and usage: XNLI has become a standard benchmark for cross-lingual representation learning and multilingual natural language inference research. It is widely used to assess multilingual encoders and transfer methods, including approaches based on multilingual embeddings, translation-based strategies, and cross-lingual alignment.

Construction and availability: The development and test sets are translations of MNLI’s corresponding portions, produced to maintain label consistency across languages. The dataset is publicly available under permissive terms through FAIR and associated repositories, with English training data and multilingual dev/test sets.