UK AI institute finds benchmarks underestimate what AI agents can actually do

What happened

The UK's AI Security Institute tested frontier AI models across seven benchmarks and found that fixed evaluation budgets systematically underestimate agent capabilities. Performance improves significantly when models are given higher token budgets—in cybersecurity tasks, about 8 percent of tasks were only solved when the budget exceeded 10 million tokens, and some required 50 million. On software engineering tasks, success rates jumped about 25 percent when the token budget went from one million to ten million.

Why it matters

Standard benchmarks with low compute budgets measure the floor, not the ceiling, of what these AI systems can do. Test scores that skip higher budgets can skew decisions about deployment and risk assessment. Since token costs are falling, capabilities once thought unaffordable could become cheaper to reach over time, making it increasingly important to measure performance across different compute levels rather than at a single fixed point.

What to watch

The institute found that newer models benefit far more from extra compute than older ones, and frontier AI progress may be moving faster than benchmarks suggested—the doubling rate of AI time horizon at higher budgets is steeper than at fixed 2.5 million token budgets. AISI now tests models at multiple budgets using "minimum informative budgets" to determine when a model's reach stops growing with extra compute.