No AI model passes 60% on visual perception benchmark

PerceptionBench addresses a fundamental gap in how the AI industry has been evaluating multimodal models. The existing landscape includes 42 open-source perception benchmarks, yet the authors found they show little overlap in their error profiles—each covers a different subset of visual weaknesses. This fragmentation meant no single test or small group could capture visual perception as a whole. By building a taxonomy from actual model errors rather than theory, the benchmark reveals what previous evaluations have obscured: the specific visual skills where frontier models systematically fail. The finding that many so-called reasoning errors actually originate in the perception stage has immediate implications for how AI teams should debug and improve their systems. If a model fails a multi-step visual task, the error may not lie in the logical reasoning that follows, but in the foundational step of accurately reading the image itself.

The category-level divergence is particularly telling. Models like GPT-5.6 Sol and Gemini 3.5 Flash have vastly different overall rankings, yet their hallucination performance differs by nearly 24 percentage points in opposite directions—Sol weak, Flash comparatively strong. This pattern suggests that visual perception is not a monolithic capability but a collection of distinct sub-skills, and current frontier models have uneven strength across them. The body notes this reflects a broader pattern: a separate study using the BabyVision benchmark showed that frontier models fail at basic visual tasks tied to early childhood development (line tracing, counting hidden blocks), with Gemini 3 Pro at 49.7 percent versus a human baseline of 94.1 percent. The authors attribute this gap to a verbalization bottleneck where visual information is translated into language and loses fidelity—a constraint that may explain why even the most capable models cannot break 60 percent on PerceptionBench.