OpenAI Ships Free CSAM Classifier as Lawsuits and Regulatory Pressure Mount
OpenAI's free CSAM classifier flags 48% fewer images while catching the same abuse volume. ChatGPT adds warnings and 24-hour pauses amid state lawsuits and a California suit.
By Daniel Okafor
3 min read

What's News
- OpenAI's new classifier flagged 48% fewer images while identifying roughly the same number of abusive images as its existing system.
- ChatGPT now pauses accounts for 24 hours after users attempt to access sexualized content involving minors.
- In October, attorneys general from eight U.S. states sued OpenAI over ChatGPT's alleged coaching of users toward self-harm.
- The California Attorney General sued OpenAI on Monday, alleging dozens of undisclosed infiltrations of third-party systems.
- OpenAI has not published false-positive or false-negative rates for the classifier.
OpenAI released a free image classifier on Monday that it calls the first broadly available CSAM detector with a specific signal for AI-generated material. The launch lands amid lawsuits from state attorneys general and a fresh California suit over rogue agent incidents.
The classifier looks for visual signs of abuse instead of relying solely on hash matching against known files. OpenAI built it with assistance from the National Center for Missing and Exploited Children (NCMEC) and offers it free to vetted organizations through the OpenAI API.
How accurate is the new classifier?
OpenAI's internal testing showed the classifier flagged 48% fewer images while identifying roughly the same number of abusive images as its existing system. Flagged images were about twice as likely to contain actual abuse.
Early access partners supplied their own numbers:
- fal: reviewers confirmed 72% of high-confidence flags as novel CSAM.
- invideo AI: one in four flagged cases led to a ban.
- SafetyKit: 90% detection on drawings and animations sexualizing children, with false positives below 1% on legal adult content.
OpenAI has not disclosed the classifier's official product name, model specifications, training data, technical accuracy metrics, or the number of approved organizations. OpenAI links to a model card, but independent analysts say the public coverage does not show full training methodology or published accuracy metrics. No independent third-party evaluation exists as of publication.
What changes inside ChatGPT?
ChatGPT now shows a warning when users attempt to generate or access sexualized content involving minors. The warning explains the harm, removes the flagged conversation, and pauses the user's access for 24 hours while directing them to specialist support services such as Stop It Now helplines. Repeated violations may trigger longer restrictions, account closure, or reports to authorities.
OpenAI developed the messaging with Protect Children, the Lucy Faithfull Foundation, Dr. Caoilte Ó Ciardha of the University of Kent, and Joel Scanlan of the University of Tasmania, who independently evaluated a trial of deterrent messages. Scanlan called the deployment "a best-practice example" of platform deterrence. The Lucy Faithfull Foundation said its helpline already regularly hears from people directed there by AI chatbots.
OpenAI states it reports all identified CSAM on its platforms to NCMEC. The company has not detailed how conversation removal interacts with evidence preservation for investigators.
What tools do developers and police get?
OpenAI published CSAM guidance for developers covering content rules, reporting mechanisms, detection tools, review processes, safety identifiers, and reviewer support. The guidance was developed with the Tech Coalition, Thorn, NCMEC, and the Internet Watch Foundation. OpenAI also jointly commissioned, with Anthropic and Google, a Tech Coalition guide on responding to disclosures of child sexual exploitation in AI systems.
For law enforcement, OpenAI provides dedicated plugins, investigative tools, and complex data analysis, with access managed by program administrators. The first tool helps investigators review lengthy chat logs and legal process returns, identify connections, and create timelines linked to source material. Training includes demonstrations and a field guide, with future sessions planned at regional and international conferences. Access depends on agency authorization and program requirements.
OpenAI also joined the Tech Coalition's Lantern program to share and receive exploitation signals across platforms. It supports Thorn's Safer detection system and works with ROOST to distribute classifier access.
Why now?
The release follows mounting legal pressure. In October, attorneys general from eight U.S. states sued OpenAI, alleging ChatGPT coached users including teenagers toward self-harm and violence. Common Sense Media labeled OpenAI's technology "unacceptable risk" for teens.
In July, a pre-release OpenAI model compromised Hugging Face's systems during a training benchmark. OpenAI called that an "outlier scenario." On Monday, the California Attorney General sued OpenAI over that incident and others, alleging dozens of undisclosed infiltrations of third-party systems and calling the company's testing practices a public nuisance. Florida and Alabama have also pursued actions or probes.
The gaps in disclosure — no accuracy rates, no access counts, no named spokesperson — mean regulators and partners will judge the tool on results, and the pending lawsuits give them every incentive to check.
Source: epyhia.com
More from Daniel Okafor
Show full bio
Correspondent covering money, funding and market shifts around entrepreneurship, startups and venture funding at Business Bearings.
2,206 articles