AIToday
Image GenerationAI Safety & AlignmentTechCrunch AIPublished: Nov 17, 2026, 04:00 JST

OpenAI CSAM classifier flags 48% fewer images

Send on LINE
OpenAI CSAM classifier flags 48% fewer images

3 Key Points

  1. What happened

    OpenAI released an image classifier that detects CSAM, including AI-generated images, and is giving select companies free API access; early testing found it flags 48% fewer images while finding the same number of abusive images.

  2. Why it matters

    Fewer false flags could cut reviewers' workload, and flagged images are twice as likely to be abusive, though OpenAI says it already blocks CSAM and repeated offences may lead to permanent account removal.

WHO IT HITSTrust and safety reviewers at companies given API access may face fewer false positives to sift through, while ChatGPT users who attempt prohibited prompts now encounter warning screens and a 24-hour suspension.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

OpenAI's announcement comes amid growing safety concerns around AI tools and after several lawsuits blaming its technology for harming users, especially children. Safety-focused nonprofit Common Sense Media labeled OpenAI's tech as "unacceptable risk" for teens, and other chatbots — specifically Grok — have been used to create nonconsensual sexualized imagery, including of children.

The classifier was developed with the National Center for Missing and Exploited Children, and OpenAI is working with Tech Coalition and ROOST to share access. Beyond the classifier, OpenAI is publishing guidelines for developers and equipping law enforcement with AI-powered features like dedicated plugins, investigative tools, and complex data analysis, with access managed by program administrators, plus training and a field guide for investigators.

OpenAI noted it formed its warning messages and guidelines with organizations and researchers, including Protect Children, the Lucy Faithfull Foundation, and Joel Scanlan at the University of Tasmania, who independently evaluated a trial of deterrent messages. The company said that in early testing, the developer guidelines helped a number of developers adopt child safety tooling and register to report CSAM to NCMEC. This follows changes over the last 12 months including a safety blueprint, updated content guidelines, model retuning, and a special version of ChatGPT for teens.

FAQ
Who can use OpenAI's new CSAM classifier?
Select companies can access it through OpenAI's API at no cost, and OpenAI is working with Tech Coalition and ROOST to share access.
What happens if a ChatGPT user tries to generate content that sexualizes children?
ChatGPT shows an alert, directs the user toward specialized support, and suspends the account for 24 hours as a warning; repeated offences could lead to permanent removal.
What did early testing of the classifier show?
With partners, the classifier flagged 48% fewer images while finding the same number of abusive images, and flagged images were twice as likely to be abusive.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleOpenAI opens models to Apollo Research, METR, Transluce, Goodfire