AIToday
Large Language ModelsAI WeeklyPublished: Oct 23, 2026, 22:00 JST

OpenDS-World: GPT-6 Astra Hits 39.0% vs Humans' 21.1%

Send on LINE
OpenDS-World: GPT-6 Astra Hits 39.0% vs Humans' 21.1%

3 Key Points

  1. What happened

    OpenDS-World, a benchmark submitted to arXiv on October 19, 2026, tested twelve frontier models on 200 live-web research tasks; GPT-6 Astra led at 39.0% accuracy, while the human per-attempt baseline was 21.1% across 1,000 independent trials.

  2. Why it matters

    Because OpenDS-World runs on the live, unmodified web rather than frozen snapshots, the result is harder to attribute to training-data contamination, and it puts the top model 17.9 percentage points above the human per-attempt baseline.

WHO IT HITSAI research teams who evaluate agentic search systems now have a contamination-resistant benchmark; enterprises deploying web-research agents may see the same tool-coordination and long-horizon search failures the paper flags across all twelve models.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

OpenDS-World arrives as a direct response to a methodological problem in agent evaluation: most benchmarks rely on frozen web snapshots, which leaves open the possibility that a model's gains reflect training-data contamination rather than genuine research ability. By requiring agents to browse the live, unmodified web, locate evidence across multiple sources and file formats, and return unique factual answers, the benchmark closes that loophole. The reported gap is substantial—39.0% for GPT-6 Astra against a 21.1% human per-attempt baseline measured over 1,000 independent trials—and the shared tool environment with a common search budget means the twelve frontier models were compared under the same constraints. The paper's finding that tool coordination and long-horizon search management remain persistent failure modes across all twelve models suggests the bottleneck is not raw model capability alone but the ability to manage extended, multi-step searches. For teams building or buying web-research agents, that points to where engineering effort may be most needed.

FAQ
How many models were tested on OpenDS-World?
Twelve frontier models were evaluated in a shared tool environment with a common search budget.
What did the authors analyze beyond accuracy?
The team analyzed tool usage patterns, search costs, termination behavior, and failure recovery across all twelve models.
Why not use a frozen web snapshot?
Frozen snapshots make it possible to attribute AI gains to training data contamination; the live web removes that objection.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleDecomp community site shuts down as AI speeds ports