DiffusionGemma
A multimodal Gemma 4 model that writes text by denoising blocks of tokens.
DiffusionGemma is an open-weights generative model from Google DeepMind, built on the 26B A4B Mixture-of-Experts Gemma 4 architecture. Instead of writing one token at a time, it denoises blocks of tokens in parallel using discrete diffusion, aiming for faster text generation while taking text, image and video input.
- Generates text by denoising blocks of tokens with discrete diffusion
- 26B total parameters, 3.8B active, 8 of 128 experts plus 1 shared
- Context windows up to 256K tokens
- Google DeepMind claims over 1100 tokens per second per user on H100 in FP8
- Open weights under Apache 2.0
What it is
DiffusionGemma is a generative model from Google DeepMind, released as open weights under Apache 2.0. It is built on the 26B A4B Mixture-of-Experts Gemma 4 architecture, meaning 26 billion parameters in total with about 3.8 billion active for each token. The model takes text, images and video and produces text. Its main difference from ordinary language models is how it writes: rather than emitting one token after another, it denoises a whole block of tokens at once using discrete diffusion. Google DeepMind says this is meant to raise generation speed while keeping the model small enough to run on a single accelerator.
Inside the model
The model uses an encoder-decoder layout. An autoregressive encoder reads the prompt and stores it as a KV cache, the key-value pairs attention layers reuse. A decoder then applies bidirectional attention over a block of tokens called a canvas, reaching the cached context through cross-attention. Generation is block-autoregressive: the sampler denoises a full canvas in parallel, the finished canvas is passed through the encoder and appended to the cache, and the next canvas begins. The sparse MoE runs 8 active experts out of 128 total plus one shared. A vision encoder of roughly 550M parameters handles images. Recommended sampling uses Entropy-Bounded Denoising with up to 48 denoising steps and adaptive stopping.
What it can do
Google DeepMind reports that parallel denoising of 256 tokens yields 15 to 20 tokens per forward pass and, at low batch size on an H100 in FP8, over 1100 tokens per second per user. Simpler prompts need fewer denoising steps. The model supports thinking mode, 256K-token contexts, function calling, coding, and image and video understanding including OCR and document parsing. It is pre-trained on over 140 languages and supports 35+ out of the box. Reported scores trail Gemma 4 26B A4B on most listed benchmarks, for example 69.1% on AIME 2026 without tools against 88.3%, though it is higher on HLE without tools, 11.0% against 8.7%.
How to use it
Weights are published on Hugging Face as google/diffusiongemma-26B-A4B-it under Apache 2.0. The model loads through Transformers with DiffusionGemmaForBlockDiffusion and AutoProcessor. Guidance covers diffusion sampling settings, thinking-mode control tokens, multi-turn history handling, prompt ordering of images before text, and a visual token budget of 70, 140, 280, 560 or 1120 tokens per image.
Questions people ask about DiffusionGemma
How is DiffusionGemma different from a normal language model?
Ordinary models write one token at a time. DiffusionGemma denoises a whole block of tokens, called a canvas, in parallel using discrete diffusion, then repeats the process for the next block. Google DeepMind says this removes the sequential bottleneck and speeds up decoding.
How large is it?
The model has 26 billion total parameters in a sparse Mixture-of-Experts design, with about 3.8 billion active per token. It runs 8 active experts out of 128 total plus one shared, and includes a vision encoder of roughly 550 million parameters.
What can it take as input?
Text, images and video, in any mix within one prompt, and it outputs text. Images can vary in aspect ratio and resolution, with a visual token budget from 70 to 1120 tokens. Video is handled as sequences of frames.
How do I run it?
Weights are on Hugging Face under Apache 2.0. Install transformers, torch and accelerate, then load the model with DiffusionGemmaForBlockDiffusion and AutoProcessor using the ID google/diffusiongemma-26B-A4B-it.
Is it better than Gemma 4 26B A4B?
On the benchmarks listed, DiffusionGemma scores below Gemma 4 26B A4B on most tasks, including AIME 2026 and GPQA Diamond, but above it on HLE without tools. Its stated advantage is decoding speed rather than benchmark accuracy.
Good reading and sources
There is no English Wikipedia article on DiffusionGemma yet, so the facts on this page come from the sources listed above: huggingface.co/google/diffusiongemma-26B-A4B-it. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.