Meta contractors created thousands of fake minor accounts and sent over 45,000 crisis-related prompts to ChatGPT, Gemini, and Character.AI without the companies' knowledge as part of a safety testing project. This follows growing concern about teen deaths linked to AI chatbots and widespread underage use despite age requirements; the incident raises questions about how companies should test AI safety responsibly and whether Meta's data handling was appropriate.
Summaries like this, in your inbox every morning.
Sign up free →What happened
Meta contractors created fake accounts posing as minors under 18 and sent over 45,000 prompts about self-harm, eating disorders, and drugs to ChatGPT, Gemini, and Character.AI in August 2025. The internal project, called "Cannes," was run by contractor Covalen and remained active through at least April 2026. The tested companies were unaware of the testing.
Why it matters
Several teen deaths have been linked to AI chatbots, including a 14-year-old Character.AI user who took his own life and a 23-year-old who died by suicide after ChatGPT validated suicidal thoughts. A UK study found 64 percent of children aged 9–17 have already used AI chatbots despite weak age verification. Meta's testing appears to be an attempt to evaluate safety risks, though documents do not show what the company did with the collected data.
What to watch
Character.AI said the testing violated its terms of service. OpenAI is investigating the matter, and Google said it did not approve the tests. Meta defended the practice as responsible, industry-standard safety testing and stated it did not use the responses to train its own AI models.
No comments yet. Be the first to share your thoughts!
Log in to join the discussion
Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.
Get Started FreeFree · takes 30 seconds · unsubscribe anytime
1 minute a day. The AI essentials.
200+ sources · Email / LINE / Slack
