Google converts Gemma 4 to diffusion model, halves training with 50% speed gain

Google DeepMind's DiffusionGemma represents a shift in how text models can be adapted rather than built from scratch. The team started with Gemma-4-26B-A4B, an existing model, and converted it into a diffusion model—one that generates text by iteratively refining noisy predictions into clear output—using less than ten percent of the original training token budget.

The conversion process unfolds in two stages. First, the model learns to reconstruct noisy text blocks from example data, establishing a foundation for denoising. The second stage combines reinforcement learning (traditionally used to improve answer quality) with sampler distillation (which reduces the number of compute steps needed). Google calls this merged approach SD·RL. Together, these stages raise quality on reasoning benchmarks by an average of ten points while nearly quadrupling the number of tokens the model can generate per compute step. As a practical consequence, DiffusionGemma's answers run about 50 percent shorter, which further accelerates output.

The model's core advantage lies in its bidirectional reasoning process. Standard autoregressive language models commit to the first token of an answer before working through the full reasoning—creating situations where the model must append corrections after realizing its initial direction was wrong. In one example from Google's report, Gemma 4 begins a math problem with "-1," later realizes the correct answer is "-25," and tacks on a correction. DiffusionGemma develops the answer and reasoning in parallel, allowing it to fix mistakes before finalizing output. This capability is particularly powerful for tasks with high interdependencies: after minimal fine-tuning, DiffusionGemma solves close to 85 percent of Sudoku puzzles correctly, while the base model fails entirely at the task. Structured outputs like JSON or code repairs complete in just two to three refinement steps because the input constraints already determine most tokens. The model retains its ability to generate text word by word, letting users switch between bidirectional and autoregressive modes depending on the task.

Performance benchmarks reveal a clear speed-versus-quality trade-off. DiffusionGemma outputs text at about 1,500 tokens per second, delivering several times the output speed of both Gemma 4 and previous diffusion models while maintaining comparable accuracy. However, it trails Gemma 4 on absolute quality benchmarks. Google identifies several reasons: DiffusionGemma was retrofitted after Gemma 4 was fully trained rather than designed as a diffusion model from the start, the subsequent training phase was relatively short, and SD·RL prioritized speed over peak quality. The architecture, training data, and other settings were also carried over from Gemma 4, which may not be ideal for diffusion. The model occasionally gets stuck in repetition loops, producing individual words multiple times in a row—an artifact of aggressively reduced compute steps—and on multimodal tasks sometimes forgets to close its reasoning section properly, artificially dragging down benchmark scores.

The speed advantage holds mainly for single-user scenarios. Once about 32 concurrent requests hit the model, standard autoregressive language models catch up on throughput, limiting DiffusionGemma's practical edge in shared deployment. Google explicitly calls DiffusionGemma experimental and says the release is intended to accelerate research on text diffusion while giving the community a foundation for specialized, resource-efficient adaptations. The model is already in use: startup Interfaze deployed it for multilingual speech recognition, and a research project applied it to interactive radiology report generation. Google released it under an Apache 2.0 license on Hugging Face. Its predecessor, Gemini Diffusion, was demoed in May 2025.