Gopher
A 280 billion parameter Transformer language model from DeepMind, studied for how scale changes ability.
Gopher is a Transformer-based language model with 280 billion parameters, built by researchers at DeepMind and described in a paper first posted in December 2021. It was trained as part of a study that compared models from tens of millions of parameters up to Gopher's size on 152 tasks, to see where scale helps and where it does not.
- 280 billion parameters, the largest model in its training family
- Evaluated on 152 diverse tasks
- Scale helped reading comprehension, fact-checking and toxicity detection most
- Logical and mathematical reasoning gained less from scale
- Paper first posted 8 December 2021, revised 21 January 2022
What it is
Gopher is a large language model built on the Transformer design, the now standard architecture for systems that predict text one token at a time. It is the biggest entry in a family of models that the authors trained at many different sizes, starting from a few tens of millions of parameters and rising to Gopher's 280 billion. The point of the exercise was measurement rather than a product launch: the team ran every model in the family across 152 tasks so they could chart how performance changes with scale. Gopher is described as a research model, and the paper reports where bigger size pays off and where it does not.
How it came about
The work was carried out by a large team at DeepMind, with Jack W. Rae and Sebastian Borgeaud among the lead authors and Demis Hassabis, Koray Kavukcuoglu and Geoffrey Irving listed at the end of the author list. The paper, titled 'Scaling Language Models: Methods, Analysis & Insights from Training Gopher', was submitted to arXiv on 8 December 2021 and revised on 21 January 2022. It appeared under the computation and language category, with an artificial intelligence cross-list.
Inside the model
Gopher follows the Transformer architecture, which uses attention to weigh earlier words when predicting the next one. The authors trained a whole ladder of models rather than a single system, so that the effect of size could be isolated: the smallest had tens of millions of parameters, the largest, Gopher, had 280 billion. The paper also examines the training dataset itself and looks at how model scale interacts with bias and toxic language. It discusses applying language models to AI safety and to reducing harms in downstream uses.
What it can do
Across 152 diverse tasks, the models reached top results on the majority, according to the paper. The authors report that gains from scale were largest in reading comprehension, fact-checking and spotting toxic language. Logical and mathematical reasoning benefited less from added size. The paper also analyses how bias and toxicity change as models grow.
How to use it
Gopher is presented as a research model in a published paper. The paper describes the model and its evaluation, and discusses its use in AI safety work.
Safety and testing
The paper treats safety as part of the study itself. It analyses the training data and the model's behaviour, looking at the overlap between scale, bias and toxicity, and it discusses how language models can support AI safety and reduce downstream harms.
Why it matters
Gopher became a reference point for studying how language model performance changes with size. The paper's method of training a ladder of models and testing them on a wide task set gave later work a way to separate the effects of scale from other design choices.
Questions people ask about Gopher
What is Gopher?
Gopher is a Transformer-based language model with 280 billion parameters, built by researchers at DeepMind. It is the largest in a family of models trained at different sizes so the team could measure how performance changes as models grow.
Who made it and when?
It was built by a large DeepMind team led on the paper by Jack W. Rae and Sebastian Borgeaud. The paper was submitted to arXiv on 8 December 2021 and revised on 21 January 2022.
What was it built to do?
It was built for measurement. The authors trained models from tens of millions of parameters up to Gopher's 280 billion and ran them on 152 tasks to see where scale helps, and to study bias, toxicity and safety.
Where does scale help most?
According to the paper, the biggest gains from scale came in reading comprehension, fact-checking and identifying toxic language. Logical and mathematical reasoning saw less benefit from making the model larger.
Can I use Gopher?
Gopher is described as a research model in a published paper. The paper reports its design and evaluation and discusses its application to AI safety and reducing downstream harms.
Good reading and sources
There is no English Wikipedia article on Gopher yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2112.11446. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.