
What happened
Across 52 laboratory tasks in physical sciences, life sciences and engineering, Claude Fable 5.2 led 13 agents at 45.0, Claude Opus 5.5 scored 40.2 and GPT-6.1-sol 31.7.
Why it matters
The tasks test whether an AI can design its own experiments, not just run workflows or pick from preset options, and the authors say these remain hard for frontier models.
WHO IT HITSResearch teams using AI to plan lab work should expect current models to handle only a fraction of experimental design tasks, based on these scores. The authors themselves note the next step is studying where the models fail.
Summaries like this, every morning.
TwinLab-Bench arrives as a check on a broader claim: the preprint notes that the result comes the same day Anthropic has been pushing the stronger narrative that its Claude models solved two open black hole problems. On the benchmark's harder test, the picture is narrower — Claude Fable 5.2's 45.0 average is the best of 13 agents, but still below half the scale.
The setup is what makes the score meaningful. Each task pairs the tested model with its own 'TwinLab agent,' which acts as the environment: it reads the raw materials of a real laboratory, calls calculators built from its records, and returns readings for whatever experiment the model designs. The paper reports that this setup "almost eliminates readings that deviate from the real records and inconsistent answers to a repeated request" — which means the low scores are less likely to be an artifact of a simulated lab that behaves unrealistically.
The authors, Yang Wan and Linchao Zhu, frame the gap between running experiments and designing them as the point. Their stated next step is "an empirical study of where it fails," so the near-term output of this line of work is likely a map of failure modes rather than a leaderboard race.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
Researchers at the Technion, University of Haifa, and Google Research released AltRA-Test, which checks whethe…

The Nov 19 arXiv paper found ICLR critique overlap rose from 12.0% in 2026 to 21.3% in 2027, with LLM-flagged…

Feed Inc. began offering "Marugoto LLMO" on November 17, 2026, covering AI research, question design, page rew…

Co-Labo AI published the first of three explainer installments on ontology on November 20, 2026, aimed at manu…

The guide maps input types to fields — images and video to computer vision (CV), text to natural language proc…

Meta's AI assistant Muse can list and unfollow Instagram accounts that don't follow you back; Khalil Roberts u…
