Gemma 4
Google DeepMind's open multimodal models in five sizes, from phones to servers
Gemma 4 is a family of open-weights language models from Google DeepMind, released in five sizes: E2B, E4B, 12B Unified, 26B A4B and 31B. They take text, images and (on three sizes) audio, and are built for reasoning, coding and agentic work, with context windows up to 256K tokens and support for over 140 languages.
- Five sizes: E2B, E4B, 12B Unified, 26B A4B and 31B
- Context windows of 128K or 256K tokens
- Pre-trained on over 140 languages, working out of the box in 35+
- 26B A4B activates 3.8B of 25.2B total parameters
- 12B Unified is encoder-free, projecting images and audio into one decoder
What it is
Gemma 4 is a set of open-weights models from Google DeepMind, published in both pre-trained and instruction-tuned forms. The family has five sizes: E2B, E4B, 12B Unified, 26B A4B and 31B. The largest dense model carries 30.7B parameters, while the smallest is built for phones and laptops. All of them accept text and images and produce text; E2B, E4B and 12B also take audio. Google DeepMind positions the range for reasoning, coding and agentic tasks, with context windows of 128K or 256K tokens and training coverage across more than 140 languages.
Inside the model
The models use a decoder-only transformer with attention that alternates between a short sliding window and full global coverage, always ending on a global layer. Global layers share keys and values and use Proportional RoPE to keep long contexts cheap in memory. E2B and E4B add per-layer embeddings, so their effective counts (2.3B and 4.5B) sit well below totals of 5.1B and 8B. The 12B Unified variant drops separate vision and audio encoders, feeding image patches and waveforms straight into the decoder through linear layers. The 26B A4B is a mixture-of-experts design with 128 experts plus one shared, activating 8 per token.
What it can do
Reported instruction-tuned scores include MMLU Pro of 85.2% for 31B and 60.0% for E2B, GPQA Diamond of 84.3% and 43.4% respectively, and LiveCodeBench v6 of 80.0% against 44.0%. On AIME 2026 without tools, 31B reaches 89.2% versus 20.8% for Gemma 3 27B. Long-context MRCR v2 at 128k averages 66.4% for 31B. Capabilities cover thinking modes, image and video understanding, OCR, handwriting, function calling and speech recognition on the audio-enabled sizes.
How to use it
Weights are released under Apache 2.0 in pre-trained and instruction-tuned versions. They run with the Transformers library via AutoProcessor and AutoModelForMultimodalLM, and reasoning is switched on with enable_thinking=True. Google DeepMind describes the sizes as spanning phones and laptops through to consumer GPUs and servers.
Questions people ask about Gemma 4
What does the 'E' in E2B and E4B mean?
It stands for effective parameters. Per-layer embeddings give each decoder layer its own small lookup table, so the effective counts are 2.3B and 4.5B while totals including embeddings reach 5.1B and 8B. This keeps the models efficient on phones and laptops.
How is the 26B A4B different from the 31B dense model?
The 26B A4B is a mixture-of-experts model with 25.2B total parameters but only about 3.8B active per token, drawing on 128 experts plus one shared. Google DeepMind says it runs close to the speed of a 4B model, making it faster for inference than the 30.7B dense 31B.
What makes 12B Unified unusual?
It removes the separate vision and audio encoders used by other Gemma 4 models. Image patches and audio waveforms are projected straight into the decoder's embedding space through lightweight linear layers, which lowers multimodal latency and lets the whole model be fine-tuned in one pass.
Which languages does it support?
Google DeepMind says the family was pre-trained on more than 140 languages and works out of the box in over 35. Multilingual OCR is among the listed image capabilities.
How do you turn on reasoning?
All Gemma 4 models have configurable thinking modes. In the Transformers workflow you set enable_thinking=True and use the processor's parse_response function to read the output.
Good reading and sources
There is no English Wikipedia article on Gemma 4 yet, so the facts on this page come from the sources listed above: huggingface.co/google/gemma-4-31B-it, huggingface.co/google/gemma-4-12B-it. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.