New GUIDE benchmark enables AI agents to understand user intent and collaborate rather than just automate GUI tasks across complex software

GUIDE dataset comprises 67.5 hours of screen recordings from 120 novice users with think-aloud narrations across 10 different software applications

Benchmark introduces three evaluation tasks: Behavior State Detection, Intent Prediction, and Help Prediction to test AI model capabilities

Shifts GUI agent paradigm from pure automation (clicks/keystrokes) toward collaborative assistance that preserves user agency and supports iterative refinement of ideas

Designed to evaluate AI models' ability to perceive user behavior, infer underlying intentions, and provide contextual assistance in open-ended tasks like PowerPoint and Photoshop