AI personality test scores shift with language, study finds

The study used a 60-item test, the Big Five Inventory-2, which measures extraversion, agreeableness, conscientiousness, negative emotionality and openness, and asked each model to answer for itself rather than imitate a human. Across the board the models described themselves favorably, as agreeable and dependable with relatively low negative emotion, so the interesting part is not the absolute scores but their movement between languages. ChatGPT-4o and DeepSeek-V3 moved in opposite directions on openness, and Gemini 2.0 Flash, despite coming from a US developer, scored higher on conscientiousness and lower on negative emotionality when answering in Chinese than in English.

The researchers could not separate the effect of training data from model design, training procedures, or differences in how translated statements were read, and they note that a developer's location alone does not reveal which cultures are reflected in the training data. They also compared AI answers with human answers and found no exact match, with the AI-human gap widest for the Chinese models, though they caution that the human samples differed by country in age and recruitment, and the Chinese sample included only university students.

In their conclusion, the team said the results show AI can be psychologically stable yet culturally inconsistent, shaped by both the training environment and the language of interaction. They argue alignment methods encourage socially desirable answers but cannot remove cultural distance or language-driven personality shifts, and they call for integrating psychological and behavioral evaluation into future development toward fairer, more context-aware and ethically consistent AI systems.