New Ego2Web benchmark enables AI agents to connect real-world vision with web tasks, filling a critical gap in multimodal AI evaluation.

Ego2Web is the first benchmark combining egocentric video perception with web agent execution, addressing limitations of current web-only agent benchmarks

The benchmark pairs first-person video recordings with online tasks requiring visual understanding, task planning, and web interaction

Current benchmarks lack grounding in users' physical surroundings, preventing evaluation of scenarios where agents must recognize real-world objects via AR glasses before completing web tasks

The benchmark uses an automatic data-generation pipeline combined with human verification to create realistic task scenarios