Aya 101
A 13-billion-parameter multilingual model that follows instructions in 101 languages
Aya 101 is an instruction-tuned multilingual language model released by Cohere Labs in February 2024. Built on the mT5-xxl design, it handles 101 languages, over half of them lower-resourced, and its weights and instruction datasets were published openly to widen access beyond dominant languages.
- Follows instructions in 101 languages, over half of them lower-resourced
- 13 billion parameters, matching the mt5-xxl architecture
- Finetuned on about 25 million samples from five dataset collections
- Weights released under Apache-2.0 with the instruction datasets
- Evaluated across 99 languages with new multilingual test suites
What it is
Aya 101 is a large multilingual model that generates text and carries out instructions across 101 languages. Cohere Labs built it and put the weights online in February 2024 alongside the paper describing it. It holds 13 billion parameters and follows the same design as Google's mt5-xxl, meaning it reads an input sequence and produces an output sequence rather than only predicting the next word. More than half of the languages it covers are classed as lower-resourced. The team's stated aim is to spread language-model progress past the small set of languages that usually receive the most attention and training data.
How it came about
The work came out of Cohere Labs with a long list of researchers, led by Ahmet Üstün and including Marzieh Fadaee, Julia Kreutzer and Sara Hooker. The paper appeared on arXiv on 12 February 2024, and the model card and checkpoints were published on Hugging Face at the same time. Both the instruction datasets and the model itself were released openly, which the authors say will let the community run its own red-teaming and research on a multilingual model of this size.
Inside the model
The model is a Transformer-style autoregressive sequence-to-sequence system, matching the mt5-xxl architecture. It was finetuned on roughly 25 million samples drawn from several collections: xP3x, the Aya Dataset, the Aya Collection, a subset of the DataProvenance collection, and ShareGPT-Command. Every source was trimmed to the 101 languages that mT5 supports. The paper reports work on how the finetuning mixture was composed and how data was pruned. Languages are grouped into higher-, mid- and lower-resourced tiers using a classification from Joshi et al. (2020).
What it can do
The team reports that Aya beats mT0 and BLOOMZ across a wide range of automatic and human evaluations, while handling twice as many languages as those models. They also built evaluation suites spanning 99 languages, covering discriminative and generative tasks, human assessment and simulated win rates on both held-out and in-distribution tasks. The paper looks at toxicity, bias and safety as well. The model card gives translation and question-answering examples in Turkish and Hindi.
How to use it
Checkpoints are downloadable from Hugging Face under an Apache-2.0 licence, which the team chose to support open multilingual work. The instruction datasets are released too. Usage runs through the standard Transformers library, loading the model as a sequence-to-sequence model with its tokenizer.
Safety and testing
Sections 6 and 7 of the paper cover the team's safety mitigation work and their benchmarking of toxicity and bias across several languages. The model card states that publishing an open multilingual model of this kind is meant to enable community red-teaming and further research.
Why it matters
The release added an openly available multilingual option at 13 billion parameters, with instruction data published alongside the weights. The authors frame it as a way to widen access to language-model progress beyond the well-resourced languages that dominate the field, and to give researchers a model they can test and build on.
Questions people ask about Aya 101
Who created Aya 101?
Cohere Labs developed it, with a research team led by Ahmet Üstün and including Marzieh Fadaee, Julia Kreutzer and Sara Hooker among the authors. The paper was posted on arXiv on 12 February 2024, with model weights released at the same time.
How big is it and what is it built on?
It has 13 billion parameters and uses the same architecture as mt5-xxl, a Transformer-style autoregressive sequence-to-sequence design. That means it takes a text input and generates a text output, rather than only continuing a prompt word by word.
What data was it trained on?
Finetuning used about 25 million samples from xP3x, the Aya Dataset, the Aya Collection, a subset of the DataProvenance collection and ShareGPT-Command. Each source was cut down to the 101 languages supported by mT5, with filtering and pruning described in the paper.
How do I use it?
The checkpoints sit on Hugging Face under an Apache-2.0 licence. You load them with the Transformers library as a sequence-to-sequence model and its tokenizer, then call generate on encoded prompts, as in the Turkish translation and Hindi question examples on the model card.
What about safety and bias?
The paper devotes two sections to safety mitigation and to measuring toxicity and bias across multiple languages. The team says releasing the model openly is intended to let the community carry out red-teaming and further research on a multilingual model.
Good reading and sources
There is no English Wikipedia article on Aya 101 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2402.07827, huggingface.co/CohereLabs/aya-101. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.