Anthropic's Opus 5 reaches 30.2% on ARC-AGI-3, crushing prior 7.8% record

Claude Opus 5's performance on ARC-AGI-3 represents a dramatic leap—jumping from 7.8 percent to 30.2 percent—but the nature of that improvement remains contested. ARC Prize credits stronger logical reasoning capabilities that enable more autonomous exploration and planning in unfamiliar environments. The model's ability to translate tasks into algebraic notation and independently formulate reflection equations marks genuinely novel behavior from a reasoning standpoint. However, Anthropic has not disclosed its training approach, leaving open the question of whether the gain reflects fundamental reasoning improvements or targeted optimization for ARC-AGI-3's puzzle-specific mechanics.

The tension between these interpretations comes into focus when comparing results across benchmarks. Independent testing on Witness, another interactive puzzle benchmark, shows Opus 5 scoring 43.4—statistically equivalent to competitors like Kimi K3 and Fable 5, with marginal gains over Opus 4.8 compared to its massive ARC-AGI-3 jump. Guanghan Ning, who designed Witness, suggests this pattern fits training on genre-specific data, though he clarifies that Opus 5 did generalize to Witness, just far less dramatically. Greg Kamradt, a researcher behind ARC-AGI-3, counters that Witness may not fully test novelty adaptation and that broader reasoning gains are not ruled out by a single weak result.

The resolution may lie in how AI benchmarking evolves. Ning draws a parallel to coding benchmarks, which moved from saturated benchmarks like HumanEval to frequently updated competitions as models improved. With ARC-AGI-3 now emerging as a major target for interactive reasoning research, it will likely attract concentrated training effort first; covering more edge cases and generalization patterns could then help models transfer those skills to wider reasoning tasks. Until independent benchmarks that test true novelty-handling mature, the question of whether Opus 5's ARC-AGI-3 lead reflects fundamental progress or focused optimization will remain open.