AIToday
Large Language ModelsAI WeeklyPublished: Nov 20, 2026, 22:00 JST

Claude Fable 5.2 tops TwinLab-Bench at 45.0 on lab design

Send on LINE
Claude Fable 5.2 tops TwinLab-Bench at 45.0 on lab design

3 Key Points

  1. What happened

    Across 52 laboratory tasks in physical sciences, life sciences and engineering, Claude Fable 5.2 led 13 agents at 45.0, Claude Opus 5.5 scored 40.2 and GPT-6.1-sol 31.7.

  2. Why it matters

    The tasks test whether an AI can design its own experiments, not just run workflows or pick from preset options, and the authors say these remain hard for frontier models.

WHO IT HITSResearch teams using AI to plan lab work should expect current models to handle only a fraction of experimental design tasks, based on these scores. The authors themselves note the next step is studying where the models fail.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

TwinLab-Bench arrives as a check on a broader claim: the preprint notes that the result comes the same day Anthropic has been pushing the stronger narrative that its Claude models solved two open black hole problems. On the benchmark's harder test, the picture is narrower — Claude Fable 5.2's 45.0 average is the best of 13 agents, but still below half the scale.

The setup is what makes the score meaningful. Each task pairs the tested model with its own 'TwinLab agent,' which acts as the environment: it reads the raw materials of a real laboratory, calls calculators built from its records, and returns readings for whatever experiment the model designs. The paper reports that this setup "almost eliminates readings that deviate from the real records and inconsistent answers to a repeated request" — which means the low scores are less likely to be an artifact of a simulated lab that behaves unrealistically.

The authors, Yang Wan and Linchao Zhu, frame the gap between running experiments and designing them as the point. Their stated next step is "an empirical study of where it fails," so the near-term output of this line of work is likely a map of failure modes rather than a leaderboard race.

FAQ
What does TwinLab-Bench actually measure?
It tests the hardest of three abilities expected of an AI scientist: designing its own experiments, rather than running a given workflow or choosing from a predefined design space. Each task pairs a model with a 'TwinLab agent' that reads a real lab's records and returns readings.
How is a model's final report scored?
The agent's final report is scored against a hidden answer taken from real laboratory records. The benchmark supports more than 100 experimental action types.
Which models were compared?
Thirteen agents were evaluated, including Claude Fable 5.2, Anthropic's Claude Opus 5.5 and GPT-6.1-sol, with DeepSeek, Qwen, Kimi and GLM systems rounding out the table.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleMuse lets Instagram users mass-unfollow non-followers