DO-Bench diagnostic benchmark isolates sources of object hallucination in vision-language models through controlled multimodal interventions.

Researchers introduce DO-Bench, a controlled diagnostic benchmark that separates errors in vision-language models (AI systems that process both images and text) into two dimensions: Prior Override (testing resistance to contextual text bias while holding visual evidence constant) and Perception-Limited (measuring visual grounding by progressively enhancing visual evidence from full-scene to localized object crops).

The benchmark defines two diagnostic metrics—PriorRobust and PerceptionAbility—to quantify how models behave when choosing between textual context and visual perception in binary object existence verification (determining whether an object is present in an image).

Evaluations across diverse open- and closed-source vision-language models reveal systematic differences in prior sensitivity and perceptual reliability, demonstrating that object hallucination reflects heterogeneous, mechanism-dependent failure patterns beyond aggregate accuracy.