Vision-language models show significant performance drops on Indian languages, with accuracy declining by up to 25 percentage points compared to English.

Researchers conducted the first cross-lingual visual reasoning audit translating 980 questions from MathVista, ScienceQA, and MMMU into six Indian languages: Hindi, Tamil, Telugu, Bengali, Kannada, and Marathi

Eight VLMs tested including GPT-4o and open-source 7B models showed accuracy drops of 9.8-25 percentage points when switching from English to Indian languages

Dravidian languages (Tamil, Telugu, Kannada) suffered up to 13.2 percentage points more degradation than Indo-Aryan languages like Hindi and Bengali

Chain-of-thought prompting backfired on Bengali (-14.4 pp) and Kannada (-11.4 pp), revealing English-centric model design and training biases