News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
MT5

MT5

A multilingual text-to-text transformer pretrained on 101 languages

mT5 is a multilingual version of the T5 text-to-text transformer, released in 2020 by a Google research team. It was pretrained on the mC4 corpus, a Common Crawl dataset spanning 101 languages, and the authors report leading results on many multilingual benchmarks, with all code and checkpoints made public.

Share on WhatsApp

  • Multilingual variant of Google's T5, released in October 2020
  • Pretrained on mC4, a Common Crawl corpus covering 101 languages
  • Ships as code and checkpoints, all publicly available
  • Must be fine-tuned on a task before use, having seen no labelled data
  • Includes a trick to stop zero-shot answers drifting into the wrong language

What it is

mT5 is a multilingual member of the T5 family of text-to-text transformers. A Google research group introduced it in a paper first posted in October 2020. Like T5, it casts every natural language processing job as generating text from text. What sets it apart is pretraining on mC4, a Common Crawl collection that spans 101 languages. The team released the code and trained checkpoints openly. Because the model saw only raw pretraining data and no labelled examples, it must be fine-tuned on a specific task before it can be used for that task.

How it came about

The work builds on T5, Google's earlier text-to-text model for English. A team of eight researchers adapted it for many languages and posted the paper on arXiv on 22 October 2020, with a later revision in March 2021. Alongside the write-up they published the code and the trained checkpoints, and the model card notes that other community checkpoints exist as well.

Inside the model

mT5 keeps the T5 shape: an encoder reads the input and a decoder writes the output, with every task framed as text generation. Its main change is the training corpus. The authors built mC4 from Common Crawl web pages so that 101 languages are covered, from Afrikaans and Amharic through to Zulu. Pretraining used this raw text only, with no supervised examples. The paper also sets out a modified training recipe and a simple trick that stops a zero-shot model from partly answering in the wrong language, a problem the authors call accidental translation.

What it can do

The authors report that mT5 reached state-of-the-art results on many multilingual benchmarks. They also describe a technique to keep zero-shot predictions in the intended language. The model card is clear about one limit: since the model was pretrained only on mC4 and never shown labelled data, it is not usable on a downstream task until it has been fine-tuned for that task.

How to use it

The code and model checkpoints were released publicly with the paper. Checkpoints are hosted on Hugging Face, including an mt5-xxl version, and the model card points to further community checkpoints. The research page lists links to the code and data as well.

Why it matters

The paper and its checkpoints are public, and the model card notes that other community checkpoints have been built from it.

Questions people ask about MT5

What is mT5?

It is a multilingual version of the T5 text-to-text transformer from Google. Pretrained on the mC4 web corpus across 101 languages, it turns every task into generating text from text and was released with public code and checkpoints in 2020.

Which languages does it cover?

The mC4 pretraining corpus spans 101 languages, listed on the model card from Afrikaans, Albanian and Amharic through to Yoruba and Zulu. It includes widely spoken languages such as English, Chinese, Spanish, Hindi and Arabic, plus many smaller ones.

Can I use it straight away?

No. The model card states that mT5 saw only pretraining data and no supervised training, so it has to be fine-tuned on a downstream task before it is usable for that task.

How do I get it?

The code and model checkpoints were published openly with the paper. Checkpoints are hosted on Hugging Face, including an mt5-xxl size, and the model card links to other community checkpoints built from the same base.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on MT5 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2010.11934, huggingface.co/google/mt5-xxl. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.