New DOVE framework uses distributional analysis to better evaluate how well large language models align with diverse cultural values worldwide

Researchers introduce DOVE, an evaluation framework that addresses three key limitations (C³ challenge) of existing cultural value benchmarks: reliance on multiple-choice formats, failure to account for subcultural diversity, and mismatch with real-world open-ended generation

DOVE constructs a compact value-codebook from 10,000 documents using rate-distortion variational optimization to map text into a structured value space while filtering semantic noise

The framework measures alignment using unbalanced optimal transport to capture both intra-cultural distributional structures and sub-group diversity

Testing across 12 different LLMs demonstrates that DOVE achieves superior predictive validity compared to existing benchmarks for assessing cultural value alignment