Meta contractors posed as minors to test rival AI chatbots on harmful topics

What happened

Hundreds of contractors working for Meta, managed by contractor Covalen, created fake under-18 accounts and sent over 45,000 prompts to OpenAI's ChatGPT, Google's Gemini, and Character.AI between August 2025 and April 2025. The prompts, designed to test how the chatbots handled requests about suicide, self-harm, eating disorders, sex, drugs, and other high-risk subjects, included images of pills, knives, and nooses. The companies being tested were not aware of the effort.

Why it matters

The testing appears to violate the terms of service of all three competitors—OpenAI bars unsolicited safety testing and efforts to bypass safeguards, Google prohibits attempts to bypass safety filters outside its authorized programs, and Character.AI prohibits harmful and exploitative content. Meta framed the work as routine "comprehensive AI safety benchmarking," but safety experts and former contractors noted that the scale, secrecy, use of dummy child accounts, and blending of safety evaluation with competitor benchmarking raised concerns about whether it amounted to gathering competitive intelligence under a safety guise.

What to watch

OpenAI said it is "looking into the issue," Character.AI confirmed the testing violated its terms and policies, and Google said it did not authorize the testing and lacks sufficient information to determine whether it violated its terms. The incident highlights ambiguity around what constitutes acceptable AI safety evaluation versus covert competitor testing.