Google converts Gemma 4 to diffusion model, halves training with 50% speed gain

Google's approach to DiffusionGemma sidesteps the conventional assumption that building a new model requires training from scratch. By retrofitting an existing, well-trained model (Gemma 4) into a diffusion architecture, the team achieved substantial efficiency gains—using less than ten percent of the original training tokens while nearly quadrupling tokens per compute step. The two-stage training strategy merges reinforcement learning (which typically boosts quality) with sampler distillation (which reduces compute steps), creating a unified process that raises reasoning benchmark scores by an average of ten points.

The model's bidirectional reasoning capability—developing answer and reasoning in parallel rather than committing to outputs sequentially—addresses a fundamental constraint of autoregressive language models. This enables self-correction before finalization, as demonstrated by its ability to solve close to 85 percent of Sudoku puzzles after minimal fine-tuning, whereas the base model fails entirely. Structured outputs like JSON or code repairs complete in just two to three refinement steps because the input already constrains most tokens. However, DiffusionGemma's practical advantages come with trade-offs: it trails Gemma 4 on absolute quality benchmarks, occasionally produces repetition loops due to aggressively reduced compute steps, and maintains its speed edge primarily in single-user settings (once about 32 concurrent requests arrive, standard language models catch up on throughput).

Google's explicit framing of DiffusionGemma as experimental reflects this balance. The release aims to accelerate research on text diffusion and provide a foundation for community adaptations—already evident in its adoption by Interfaze for multilingual speech recognition and in radiology report research. The model's architecture, training data, and settings were carried over from Gemma 4, which Google acknowledges are not necessarily ideal for diffusion, suggesting room for specialized optimization.