News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Flan-T5

Flan-T5

Instruction-finetuned T5 checkpoints released by Google for zero-shot and few-shot work

Flan-T5 is a set of language model checkpoints from Google, released in October 2022 alongside the paper "Scaling Instruction-Finetuned Language Models". Each T5 model size was further trained on instructions covering over 1000 tasks and more languages, and the authors say the resulting models give strong few-shot results even against much larger systems.

Share on WhatsApp

  • Released 20 October 2022 with the paper Scaling Instruction-Finetuned Language Models
  • One instruction-finetuned model per T5 size, from small up to XXL
  • Fintuned on a mixture covering more than 1000 tasks and extra languages
  • Trained on Google Cloud TPU v3 or v4 pods with the t5x codebase and JAX
  • Authors say few-shot results hold up against much larger models such as PaLM 62B

What it is

Flan-T5 is a family of language model checkpoints published by Google in 2022. It takes the pretrained T5 models and continues training them on instructions, so the same model can be pointed at many different jobs by phrasing the request as a command. There is one Flan variant for each T5 size, from small to XXL. The largest, Flan-T5 XXL, is the one documented in detail. The authors position it mainly as a research tool for studying zero-shot and few-shot behaviour, reasoning, question answering, fairness and safety, rather than as a product for direct deployment.

How it came about

The work comes from a large Google research team led by Hyung Won Chung and Jason Wei, with 35 authors in total. The paper "Scaling Instruction-Finetuned Language Models" was submitted to arXiv on 20 October 2022 and revised several times, the last version dated 6 December 2022. Alongside the paper, the team publicly released Flan-T5 checkpoints, with original files hosted in the Google Research t5x repository and mirrored on Hugging Face.

Inside the model

Flan-T5 starts from pretrained T5 weights and is then finetuned with instructions, which the authors describe as a general way to improve both performance and usability of a pretrained model. The study varied three things: how many tasks were used, how large the model was, and whether chain-of-thought data was included. Training ran on Google Cloud TPU v3 or TPU v4 pods using the t5x codebase with JAX. The model card notes the training mixture covers more than 1000 tasks and more languages than plain T5. Exact parameter counts and token totals are not given in these pages.

What it can do

The authors report gains across PaLM, T5 and U-PaLM, across zero-shot, few-shot and chain-of-thought prompting, and on benchmarks including MMLU, BBH, TyDiQA, MGSM and open-ended generation. Their headline figure is Flan-PaLM 540B reaching 75.2% on five-shot MMLU, beating PaLM 540B by 9.4 points on average after finetuning on 1.8K tasks. For Flan-T5 specifically they claim strong few-shot results even next to much larger models such as PaLM 62B. Evaluation covered 1836 task and language combinations. The card states the model has not been tested in real-world applications.

How to use it

Flan-T5 checkpoints are downloadable, with the XXL version published as google/flan-t5-xxl on Hugging Face and the full set listed in the t5x repository. The model card shows how to run it in PyTorch on CPU or GPU, including half precision and 8-bit loading. The paper itself is under Creative Commons Attribution 4.0 International; no separate weight licence is stated on these pages.

Safety and testing

The model card carries warnings copied from the paper's own documentation. It says language models including Flan-T5 can be used to generate harmful text, and that the model should not be deployed without first assessing safety and fairness for the specific application. The finetuning data was not filtered for explicit content or checked for bias, so the model may reproduce inappropriate output or biases present in that data. It should not be used for cases such as abusive speech generation.

Reception and debate

The pages describe the model's own claims rather than outside reception. The authors present Flan-T5 as a public companion to the larger Flan-PaLM results, offering few-shot ability at a fraction of the size. No reviews, bans or legal challenges are recorded here.

Why it matters

The paper frames instruction finetuning as a general method that works across model families and prompting styles, and the released Flan-T5 checkpoints made that recipe available at small sizes. Because the checkpoints are openly downloadable, they became a common starting point for researchers and practitioners wanting instruction-following behaviour without a very large model.

Questions people ask about Flan-T5

What is Flan-T5?

It is a set of T5 language models that Google continued training on instructions. Each T5 size has a matching Flan version. The result follows written commands across many tasks, and the checkpoints were released publicly in October 2022 for research use.

How is it different from plain T5?

The base weights come from pretrained T5. Flan-T5 adds finetuning on a mixture of more than 1000 tasks phrased as instructions, and covers more languages. The authors say that for the same parameter count, Flan-T5 does better across the board than the original.

How large is it?

There is one Flan model per T5 size, so the family spans several scales. The best documented checkpoint here is Flan-T5 XXL. Exact parameter counts are not given on these pages, though the authors say its few-shot results compare well with PaLM 62B.

How do I run it?

Download google/flan-t5-xxl from Hugging Face and load it with the T5 tokenizer and T5ForConditionalGeneration classes. The card shows CPU and GPU examples, plus half precision and 8-bit loading options for tighter memory budgets.

What should I watch out for?

The training data was not filtered for explicit content or checked for bias, so output may be inappropriate or biased. The authors say it has not been tested in real-world use and should not be deployed without a safety and fairness review for your application.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Flan-T5 yet, so the facts on this page come from the sources listed above: huggingface.co/google/flan-t5-xxl, arxiv.org/abs/2210.11416. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.