Google updates Gemma 4 with faster inference, tool-calling fixes

Google has released a significant performance and reliability update to Gemma 4, its open-source AI model available to developers. The update is distributed across all parameter sizes—including the 31B, E4B, and the newest 12B variants—but ships under the same "Gemma 4" name, a choice that has drawn criticism from the community.

The core improvement centers on inference speed and latency. By enabling Flash Attention 4, a technique that optimizes how the model processes incoming text, Google achieves a 25 to 70 percent boost in prompt processing speed and a reduction of up to 31 percent in time to first token (the latency before the model generates its first response) on Nvidia Hopper GPUs. These gains directly translate to faster, cheaper inference for developers running the model in production.

Beyond raw speed, the update addresses two critical reliability gaps. Google fixed bugs in tool calling, the feature that allows the model to autonomously trigger external tools—essential for agentic applications where the AI makes decisions and takes actions without human intervention. The 31B variant shows particularly strong improvements in agentic reasoning and tool-calling performance across all tested scenarios, with the highest gains of 10.1 percent appearing in telecommunications use cases. Google also reduced instances of truncated or incomplete responses, a problem that would otherwise undermine user trust in longer-form outputs.

For image processing tasks, the update introduces tuning flexibility. Users can manually increase the "max_soft_tokens" parameter from its default of 280 to 1,120, enabling sharper OCR (optical character recognition) results and support for resolutions up to 2.51 megapixels. Google published an interactive configurator on Hugging Face to guide users through this adjustment without requiring deep technical knowledge. The published benchmarks focus on comparisons between the 31B and E4B variants against their predecessors, though the Hugging Face repository confirms all sizes received updates. The decision to release the update without a version bump—keeping it as "Gemma 4" rather than labeling it "Gemma 4.1"—has prompted community feedback questioning whether the scale of improvements warrants a separate version identifier to avoid confusion.

Google's Gemma 4 update represents a targeted refinement of its open model rather than a major new release. The update addresses three specific pain points: raw inference speed on Nvidia hardware, a critical reliability issue with tool calling (essential for agentic use cases), and response truncation that would undermine user experience. The performance gains—particularly the 25–70 percent speedup and 31 percent reduction in time-to-first-token—are meaningful for production deployments where latency directly affects cost and user experience.

The agentic reasoning improvements, with up to 10.1 percent gains in telecommunications scenarios, suggest the update targets enterprise workflows where the model autonomously triggers external tools. The image processing tuning option (max_soft_tokens from 280 to 1,120) indicates Google is also addressing domain-specific use cases, with the Hugging Face configurator making this accessible to non-expert users. However, the decision to ship the update under the same "Gemma 4" name has drawn pushback from the open-source community, who expected a version bump (like "Gemma 4.1") to signal the magnitude of the changes—a versioning practice that would typically help developers track breaking changes or significant improvements.