
What happened
OpenDS-World, a benchmark submitted to arXiv on October 19, 2026, tested twelve frontier models on 200 live-web research tasks; GPT-6 Astra led at 39.0% accuracy, while the human per-attempt baseline was 21.1% across 1,000 independent trials.
Why it matters
Because OpenDS-World runs on the live, unmodified web rather than frozen snapshots, the result is harder to attribute to training-data contamination, and it puts the top model 17.9 percentage points above the human per-attempt baseline.
WHO IT HITSAI research teams who evaluate agentic search systems now have a contamination-resistant benchmark; enterprises deploying web-research agents may see the same tool-coordination and long-horizon search failures the paper flags across all twelve models.
Summaries like this, every morning.
OpenDS-World arrives as a direct response to a methodological problem in agent evaluation: most benchmarks rely on frozen web snapshots, which leaves open the possibility that a model's gains reflect training-data contamination rather than genuine research ability. By requiring agents to browse the live, unmodified web, locate evidence across multiple sources and file formats, and return unique factual answers, the benchmark closes that loophole. The reported gap is substantial—39.0% for GPT-6 Astra against a 21.1% human per-attempt baseline measured over 1,000 independent trials—and the shared tool environment with a common search budget means the twelve frontier models were compared under the same constraints. The paper's finding that tool coordination and long-horizon search management remain persistent failure modes across all twelve models suggests the bottleneck is not raw model capability alone but the ability to manage extended, multi-step searches. For teams building or buying web-research agents, that points to where engineering effort may be most needed.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
In a federal case in the U.S

Anthropic launched an "Anthropic managed" version of Claude on Google Cloud on the 22nd (US time), giving main…

On October 13 an identifier, claude-fable-5-5, was reported in Microsoft Azure's East US 2 model catalog with…

Microsoft said on October 21 that attackers tied to China's government are automating intrusions with ARTEX, a…

1Password unveiled 1Password for Agents, which lets an AI agent request credentials from a user's vault and, o…

Developer Simon Edgerson published WorldTUI, which uses a roughly 360M-parameter model to predict the next ter…
