kazanlm
KazanLM is a family of transformer-based language models developed by researchers affiliated with Kazan University and partners. The project seeks to advance multilingual natural language processing by providing scalable tools that can be used for research, education, and industry. The models are released in several sizes and are available under an open-source license.
KazanLM uses a decoder-only transformer architecture. Available versions span from roughly 2 billion to 20 billion
The models support a range of tasks, including instruction following, text generation, summarization, translation, and code
Evaluations show competitive performance for models in the 2–10 billion parameter range on several standard benchmarks,
See also: transformer (machine learning), large language model, multilingual NLP.