
What happened
METR released Time Horizon 1.1, expanding its benchmark from 170 to 228 tasks and scoring judgment-heavy work for the first time. No model produced an AI evaluation METR employees considered useful; Opus 5.5 reached only "borderline above the bar."
Why it matters
Models still double their task time horizon roughly every seven months on coding and math, but they fall below human performance on judgment tasks, suggesting "research taste" could be a major bottleneck for automated AI research.
WHO IT HITSAI research teams and benchmark designers who rely on METR's time-horizon charts to forecast progress now face a clearer split: strong coding and math gains, but weak judgment that may limit automation of research itself.
Summaries like this, every morning.
METR's update arrives as frontier agents keep showing a consistent shape: strong on bounded technical tasks, brittle on open-ended ones. The doubling of time horizons every 3.6 months on verifiable technical work stands in contrast to the judgment side, where models landed below human performance on scoring philosophical arguments — though METR staff themselves disagree about how relevant that test is to automating AI research.
The benchmark's expansion to 228 tasks and its first formal scoring of judgment-heavy work mark a shift in what is being measured, not just how well models perform. Computer scientist Sayash Kapoor framed the stakes, arguing that "research taste" — choosing promising questions, hypotheses and experiments — "could be a major bottleneck" on the path to automated AI research. METR's own caution that results above 16 hours remain unreliable suggests the plot's most eye-catching numbers sit inside the error bars rather than above them, a reliability issue that compounds as task lengths grow.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
The Elders published a "Humanity Threshold" declaration dated 16 July 2026 calling for a treaty that would bar…

Google search VP Robby Stein confirmed on X that the redesigned desktop page at Google.com is now official aft…

In its 2026 sustainability report, Meta said it is re-evaluating the 2030 timeframe for its net-zero goal, aft…

Meta is rethinking its climate goal after emissions rose 45%, a jump tied to the energy demands of AI, Axios r…

American Banker published exclusive research examining how JPMorgan's AI strategy is putting governance to the…

Home Depot says its Magic Apron AI assistant now fields millions of questions per month and is trained on prod…
