AIToday

OpenAI launches GPT-5.6 in three sizes, beats Claude on agent benchmarks

Simon Willison's Weblog2h ago
OpenAI launches GPT-5.6 in three sizes, beats Claude on agent benchmarks

Key takeaway

OpenAI's new GPT-5.6 family of three models is now available, with the largest version (Sol) scoring 53.6 on the Agents' Last Exam professional-workflow benchmark, exceeding Claude Fable 5 by 13.1 points. Smaller models Terra and Luna also beat Fable 5 performance at approximately one-sixteenth the cost, offering businesses a more efficient way to deploy agentic AI for long-running tasks.

Summaries like this, in your inbox every morning.

Sign up free →

3 Key Points

  • What happened

    OpenAI released three models of its GPT-5.6 family—Luna, Terra, and Sol (smallest to largest)—with general availability starting today. On the Agents' Last Exam benchmark, GPT-5.6 Sol scored 53.6, beating Claude Fable 5's 53.6-point lead by 13.1 points, while Luna and Terra outperformed Fable 5 at around one-sixteenth the estimated cost.

  • Why it matters

    The new models deliver agentic performance (long-running professional workflows) more efficiently than existing competitors. Smaller models like Terra and Luna achieving Fable 5-level results at a fraction of the cost may appeal to businesses looking to deploy AI agents in real-world applications without prohibitive spending.

  • What to watch

    Pricing is Luna $1/$6, Terra $2.50/$15, and Sol $5/$30 (per 1M input/output tokens). New API features include Programmatic Tool Calling (for orchestrating tool calls), Multi-agent (spinning up subagents in parallel), and Prompt cache breakpoints (explicit cache control). OpenAI also published findings that approximately 30% of SWE-bench Pro tasks are broken, after Claude Fable 5 outperformed GPT-5.6 Sol on that benchmark (80% vs. 64.6%).

Context & Analysis

OpenAI's release of GPT-5.6 marks a direct challenge to Claude's market position in agentic AI, the category of AI systems that autonomously execute long-running professional tasks. The benchmark results reveal a strategic focus: while Sol dominates on Agents' Last Exam (the company's flagship agentic benchmark), the real competitive advantage may lie in the smaller models. Terra and Luna achieving performance parity with Claude Fable 5 at dramatically lower cost—around one-sixteenth the expense—addresses a practical constraint that has limited AI deployment in production systems: operational expense. The cost-efficiency gain extends across reasoning levels, suggesting OpenAI has improved its token utilization broadly.

However, the story is more nuanced than a clean win. OpenAI's own audit of SWE-Bench Pro (a coding task benchmark) revealed that approximately 30% of tasks are broken, a finding that undermines the very benchmark where Claude Fable 5 substantially outperformed GPT-5.6 Sol. This suggests OpenAI is being transparent about benchmark quality even when the results favor competitors—though the timing of publishing this critique immediately before a product launch where an alternative benchmark favors the new model may be read as strategic framing.

The new API features—particularly Programmatic Tool Calling and Multi-agent—signal that OpenAI is embedding agentic patterns directly into the core platform rather than leaving them to third-party orchestration. This moves the competition away from raw model capability and toward the developer experience of building multi-step, parallel AI workflows, a shift that could accelerate production adoption if developers find the abstractions genuinely useful.

FAQ

What are the prices for GPT-5.6 models?
Luna costs $1/$6 per 1M input/output tokens; Terra is $2.50/$15; Sol is $5/$30. For comparison, Claude Opus is $5/$25 and Claude Fable 5 is $10/$50.
How does GPT-5.6 Sol compare to Claude Fable 5 on benchmarks?
On Agents' Last Exam, GPT-5.6 Sol scored 53.6 versus Claude Fable 5's score of 40.5, a 13.1-point lead. However, Claude Fable 5 achieved 80% on SWE-bench Pro compared to GPT-5.6 Sol's 64.6%.
What new API features are included?
OpenAI added Programmatic Tool Calling (compose and run JavaScript to orchestrate tool calls), Multi-agent (spin up subagents for parallel work), Prompt cache breakpoints (explicit cache control), and detail: original on image requests (to avoid resizing before processing).

Discussion

No comments yet. Be the first to share your thoughts!

Log in to join the discussion

Related Articles

Stay ahead with AI news

Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.

Get Started Free

Free · takes 30 seconds · unsubscribe anytime

1 minute a day. The AI essentials.

200+ sources · Email / LINE / Slack

Get it free →