GLaM
A trillion-parameter mixture-of-experts language model from Google Research
GLaM (Generalist Language Model) is a family of language models built on a sparsely activated mixture-of-experts design, announced by Google researchers in December 2021. Its largest version holds 1.2 trillion parameters yet, the team reports, uses a third of GPT-3's training energy while scoring better overall on zero-shot and one-shot NLP tasks.
- Largest GLaM has 1.2 trillion parameters, about 7x GPT-3
- Uses a sparsely activated mixture-of-experts architecture
- Trained with roughly 1/3 of GPT-3's energy, the team reports
- Needs half of GPT-3's inference FLOPs
- Better overall zero-shot and one-shot results on 29 NLP tasks
What it is
GLaM stands for Generalist Language Model. It is a family of language models described by Google researchers in a paper first posted in December 2021. The biggest member carries 1.2 trillion parameters, roughly seven times the count of GPT-3. What makes it different is that only a small slice of those parameters is used for any given token, thanks to a sparsely activated mixture-of-experts setup. The team reports that this design reaches better overall zero-shot and one-shot results across 29 NLP tasks while spending about a third of the energy GPT-3 needed to train and half the computation for inference.
How it came about
The work was led by Nan Du with 26 co-authors from Google, including Yanping Huang, Andrew M. Dai, Quoc V Le and Zhifeng Chen. The first version of the paper appeared on arXiv on 13 December 2021, with a revision in August 2022. It arrived during a period when labs were pushing dense models ever larger and looking for cheaper ways to scale.
Inside the model
GLaM uses a mixture-of-experts architecture, meaning the network is split into many expert sub-networks and a routing mechanism picks only a few of them for each token. That keeps the active computation small even as total parameter count grows to 1.2 trillion. The authors frame this as a way to raise model capacity without the training cost of an equally large dense model. They report the largest GLaM used about a third of the energy GPT-3 consumed in training and needed half the FLOPs at inference time.
What it can do
The team evaluated GLaM on 29 NLP tasks and reports better overall zero-shot and one-shot performance than GPT-3. Zero-shot means answering with no example prompts; one-shot means a single example is given. The paper also stresses efficiency: the same results were reached with substantially less training energy and lower inference cost than a dense model of comparable standing.
How to use it
GLaM was published as a research paper. No public weights, licence or hosted product are described.
Why it matters
GLaM became an early large-scale demonstration that sparse mixture-of-experts designs could match or beat dense models at a fraction of the training and inference cost. It is part of the wave of work that pushed later labs toward sparsely activated architectures.
Questions people ask about GLaM
What does GLaM stand for?
GLaM stands for Generalist Language Model. It is a family of language models described by Google researchers in a paper first posted on arXiv in December 2021.
How is GLaM different from GPT-3?
GLaM uses a sparsely activated mixture-of-experts design, so only part of the network runs for each token. The largest version has 1.2 trillion parameters, about seven times GPT-3's count, yet the team reports it used a third of the training energy and half the inference computation.
How well does GLaM perform?
The researchers report better overall zero-shot and one-shot performance than GPT-3 across 29 NLP tasks. Zero-shot means answering without examples; one-shot means a single example is provided in the prompt.
Can I use GLaM today?
GLaM was released as a research paper. No public weights, licence terms or hosted product are described, so it appears to be research access only.
Good reading and sources
There is no English Wikipedia article on GLaM yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2112.06905. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.