News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
T5

T5

Encoder-decoder language model family from Google AI using a text-to-text framework.

Google AI introduced the T5 series of large language models in October 2019. These encoder-decoder transformers convert all tasks into a unified text format, enabling broad applications through pre-training and fine-tuning.

Share on WhatsApp

  • Released by Google AI in October 2019 under Apache-2.0 licence.
  • Uses an encoder-decoder transformer trained on the C4 dataset.
  • Converts all tasks into a uniform text-to-text format.
  • Includes variants like Switch Transformer and Flan-T5.
  • Encoder used as text conditioner for diffusion image models.

What it is

T5 is a family of large language models created by Google AI and released in late 2019. It treats every problem as a text-to-text conversion, meaning inputs and outputs are always strings. The system uses an encoder to read data and a decoder to write responses. This design allows the same architecture to handle diverse tasks like translation or summarization without changing the underlying structure.

How it came about

The original models launched in October 2019 with sizes ranging from small to 11 billion parameters. Subsequent updates improved performance, such as the 1.1 series which switched activation functions. Later variants included LM-adapted T5 for better language modeling and Switch Transformer, which introduced mixture-of-experts layers. Instruction-tuned versions like Flan-T5 appeared in 2022 to improve following specific prompts.

Inside the model

The architecture relies on an encoder-decoder transformer structure where both parts have equal layer counts. Training used the C4 dataset, a massive collection of internet text and code. A shared WordPiece tokenizer with a 32,000-word vocabulary processed inputs. Modifications included relative positional embeddings and bias-free layer normalization. All tasks were formatted as input text followed by output text during pre-training.

What it can do

Models can perform machine translation, text summarization, and grammatical judgment after fine-tuning. They handle corrupted text restoration and code generation. The encoder also serves as a standalone text feature extractor for other systems. Variants like ByT5 operate directly on byte sequences rather than tokens, supporting multilingual processing across different languages.

How to use it

The weights are available under the Apache-2.0 licence, allowing open use. The original implementation was written in TensorFlow. A later JAX-based re-implementation called T5X provides updated tools. Researchers can download checkpoints for various sizes, from small base models to larger XXL versions, for local experimentation.

Safety and testing

The provided materials do not describe specific safety evaluations, red-teaming exercises, or content filtering mechanisms.

Reception and debate

The model is noted for its architectural influence and adoption in research projects and subsequent derivative systems.

Why it matters

T5 established a standardized text-to-text approach that influenced many downstream models. Its encoder is used in image generation systems like Google Imagen and AuraFlow as a text conditioning component. The architecture inspired variations like UL2 and Pile-T5, demonstrating its flexibility for scaling and different training objectives.

Questions people ask about T5

What is the main difference between T5 and other models?

T5 converts every task into a text-to-text format. Instead of separate heads for classification or regression, it generates text outputs for all problems, simplifying the architecture.

How does T5 process input text?

It uses a shared WordPiece tokenizer with a 32,000 word vocabulary. The encoder processes these tokens while the decoder generates the output sequence autoregressively.

Can T5 be used for image generation?

Yes, the T5 encoder acts as a text encoder for diffusion models. Systems like Google Imagen use T5-XXL to encode text instructions that condition the image generation process.

What datasets train T5 models?

The primary dataset is C4, a colossal clean crawled corpus of internet text and code. Some variants use multilingual data or instruction datasets like FLAN for fine-tuning.

Good reading and sources

Where this page comes from

The facts on this page come from the Wikipedia article "T5 (language model)", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.

It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-15.

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.