AI puzzles reveal blind spots, human wins

Puzzles have been central to AI development since the 1959 checkers-playing algorithm by IBM's Arthur Samuel, and now serve to reveal model limitations. While AI solves Connections puzzles near perfectly by early 2025, significant gaps remain.

Spatial reasoning is a key weak spot; language models cannot manipulate 3D objects like architects or mechanical engineers. Similarly, visual puzzles like ARC-AGI show models often use non-generalizable rules, unlike humans who draw on simple visual concepts.

Scale also matters: Apple researchers found models master simple Tower of Hanoi and river-crossing puzzles but falter at six or more disks or people. Logic grid puzzles from the University of Washington, Stanford, and Allen Institute show similar struggles, though commentators question whether this reflects a unique reasoning limitation or normal error as complexity increases.