Google's StreamVibe video AI boosts timing 10.7%

StreamVibe addresses a gap the researchers identify in existing video assistants: they are built for clear-cut commands like "Let me know when the water starts boiling," so they rarely pick up on unspoken intentions. They also struggle to track what's already been done, causing repeated advice or late responses, and they often stick to original instructions when users change their requests midway.

To measure the problem, the team compiled a benchmark of about 4,100 examples from more than 183 hours of everyday life and software recordings, with Gemini 2.5 Flash suggesting when to offer help and four human evaluators unanimously approving each example. The training that followed used reinforcement learning to make the model more reserved so it doesn't constantly interrupt, plus a separate memory system tracking pending requests and completed steps. While StreamVibe-VL generally beats its base model and other open-source models by a wide margin, Google's proprietary Gemini 3.8 Flash scores higher in several tests, and the specialized training does not hurt general video comprehension.

The researchers acknowledge that real-world situations are likely more complex than their test setup. The work fits a broader Google effort on AI systems that actively analyze videos, following agent-based video analysis for Gemini in September and Gemini 4 Argon's tracking of longer action sequences in robot videos.