News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Chinchilla

Chinchilla

A 70-billion-parameter DeepMind model built to test how models should scale.

Chinchilla is a family of large language models from Google DeepMind, presented in March 2022. Its 70-billion-parameter model matched or beat much larger rivals while using far less data than Gopher, showing that models should be trained on more tokens rather than simply made bigger.

Share on WhatsApp

  • 70 billion parameters trained on 1.4 trillion tokens
  • 67.5% average accuracy on MMLU, 7% above Gopher
  • Same training cost as Gopher 280B, with a quarter of the parameters
  • Scaling rule: double the size, double the training tokens
  • Used as the language model inside DeepMind's Flamingo

What it is

Chinchilla is a family of large language models created by the research team at Google DeepMind and shown in March 2022. It was built to study how such models should scale: how model size relates to the amount of training data. The best-known member has 70 billion parameters and was trained on 1.4 trillion tokens. That is a quarter of the parameters of DeepMind's earlier Gopher model but roughly four times the training data. Chinchilla was made for English text and is used mainly as a research reference point for how to train language models efficiently.

How it came about

Chinchilla follows Gopher, an earlier DeepMind family used to probe scaling behaviour. Both were trained for the same investigation, and the two cost about the same to train. The team's conclusion was that doubling a model's size should be matched by doubling its training tokens. Chinchilla applied that rule. It was presented in March 2022 and was still described as being in testing in January 2023. The same naming habit as Gopher applies: the family holds several sizes, and the largest is simply called by the family name.

Inside the model

Chinchilla is a transformer model, essentially the GPT-2 design with small changes and a different size. Like Gopher it uses RMSNorm, a normalisation step, in place of LayerNorm, and relative positional encoding rather than absolute. The one change from Gopher is the optimiser: Chinchilla is trained with AdamW instead of Adam. Its 70 billion parameters saw 1.4 trillion training tokens, against 300 billion for Gopher 280B. The authors argued that dataset scale deserves more attention, and that bigger datasets help only when the data itself is high quality.

What it can do

On the MMLU benchmark, which measures multitask language understanding, Chinchilla reached an average accuracy of 67.5%. That is 7% above Gopher's score on the same test. It was reported to outperform GPT-3. Because it is smaller than models of similar ability, it needs much less computing power to run inference and to fine-tune, which makes it easier to use downstream. It also served as the language backbone for Flamingo, DeepMind's vision-language model.

How to use it

Chinchilla was presented as a research model. It was still in testing as of January 2023. No public weights, licence or hosted product are described.

Why it matters

Chinchilla's main influence is on how later models are trained. Its scaling result, that parameter growth should be matched by token growth, pushed the field towards larger training datasets rather than ever bigger models alone. The authors also stressed data quality as a condition for that gain. The model was reused as the language component of Flamingo, DeepMind's vision-language system.

Questions people ask about Chinchilla

What is Chinchilla?

A family of large language models from Google DeepMind, presented in March 2022. It was built to test how model size and training data should grow together. The best-known version has 70 billion parameters and is trained on 1.4 trillion tokens of English text.

Why does it matter if it is smaller than Gopher?

Chinchilla matches or beats much larger models while costing about the same to train as Gopher 280B. Being smaller, it needs less computing power to run and to fine-tune, so it is easier to use in practice.

How does it differ from Gopher?

The two share the same transformer design, with RMSNorm and relative positional encoding. The change is the optimiser: Chinchilla uses AdamW where Gopher used Adam. Chinchilla also trains on four times the data at a quarter of the parameters.

How well does it score?

It reaches 67.5% average accuracy on MMLU, a multitask language understanding benchmark. That is 7% higher than Gopher on the same test, and it was reported to outperform GPT-3.

Can I use Chinchilla?

It was released as research work and was still described as in testing in January 2023. No public weights, licence terms or hosted product are described, so there is no documented way to run it yourself.

Good reading and sources

Where this page comes from

The facts on this page come from the Wikipedia article "Chinchilla (language model)", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.

It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.