Study finds Western AI models absorb Chinese censorship through training data

A peer-reviewed study published in Nature examined how state-controlled media influences the answers large language models give about China and related political topics. The researchers identified more than three million Chinese-language documents in CulturaX, an open-source training dataset used to develop and improve LLMs. They built what they describe as a "multi-part case study on China's media," focusing particularly on political subjects. Testing existing models, they found that Claude Sonnet, Claude Opus, GPT-3.5 Instruct, GPT-4, and GPT-4o had reproduced distinctive phrases from Chinese state-coordinated media at rates ranging from 3% to nearly 10%—suggesting the models had encountered and internalized the material during their training.

To isolate the causal effect, the researchers retrained Meta's Llama 2 13B model (chosen because it contained very little to zero Chinese state media in its original training) on just 6,400 Chinese state-scripted news examples. The retrained model then produced more Beijing-friendly answers than the baseline nearly 80% of the time. At higher levels of additional training, the gap widened starkly. After 64,000 state-scripted examples, when asked whether China is an autocracy, the baseline model answered yes—but the state-scripted version instead described China as democratic and invoked the Chinese Communist Party's concept of "people's democracy." The researchers could not run the same controlled experiment on proprietary systems like OpenAI's and Anthropic's, so instead they asked identical political questions in Chinese and English and compared the responses. For Claude Sonnet, the Chinese-language response was rated as more favorable to Chinese leaders and institutions 68.8% of the time; for Claude Opus, 88.2%; for GPT-3.5, 72.6%; and for GPT-4o, 84%. A broader audit across 6,051 prompts spanning 37 countries found that countries with lower press freedom tended to receive more favorable descriptions when queried in their dominant language rather than in English.

A separate audit by Meta's Oversight Board revealed a different but related problem: some Western AI models were refusing to produce content critical of leaders in authoritarian countries even when users accessed them from outside those countries. Researchers tested 10 commercial models from Anthropic, DeepSeek, Google, Meta, OpenAI and xAI, using identical prompts requesting political criticism of five restrictive-speech countries (China, Saudi Arabia, Thailand, Turkey and Cambodia) and five freer countries (the U.S., U.K., Japan, Taiwan and Chile). The tests were conducted from Australia. Across all 10 models, the average refusal rate for political criticism was 34% in restrictive countries versus 14% in freer ones. The disparities in individual models were striking. Anthropic's Claude Sonnet refused all five requests to create protest flyers criticizing Xi Jinping, Saudi Crown Prince Mohammed bin Salman and Thailand's King Vajiralongkorn, yet produced all five flyers criticizing President Trump and King Charles III. It also complied four of five times for Chile's then-president and three of five times for Japan's then-prime minister. Google's Gemini 3 Pro showed a similar pattern: it refused three of five involving Xi, four of five involving bin Salman, and all five involving Cambodia's king, with the model often citing criminal restrictions or lèse-majesté laws. Meta's open-weight Llama 4 Maverick complied with every request involving Trump, King Charles, and leaders of Japan, Chile, Taiwan and Turkey, but refused all five requests involving Xi, Thailand's king and Cambodia's king—in one case stating that criticism of government leaders could be "sensitive or illegal" in China.

The Nature researchers characterized the effect as concerning because it "severs information and opinion from their source, effectively laundering government-manipulated content into ostensibly objective text," disguising "the source of the influence and incentives of the state" and potentially "further increas[ing] the subtlety and persuasive power of state media control." The Oversight Board researchers similarly concluded that the results "show that there is a real and concerning risk that foundation models could be reflecting and further entrenching the restrictive speech norms of repressive regimes." Meta declined to comment; Anthropic, OpenAI and Google did not respond to Fortune's requests for comment. Neither study alleges that Beijing deliberately manipulated American AI makers, but together they suggest that training on openly available data from authoritarian information environments can bend model behavior without deliberate intervention.