What happened
Researchers at AI Forensics tested image editing tools on Hugging Face and found that 7 of 9 top tools easily converted clothed images of women into topless ones. In a separate test using a honeypot, they collected over 1,000 prompts in a week; 73 percent were sexual in nature, with 83 percent of those seeking to undress or sexualize submitted photos—95 percent targeting women, and 6.7 percent targeting apparent children.
Why it matters
Hugging Face, valued in the billions, hosts around 5,000 AI image models capable of creating nonconsensual intimate images of real people, yet appears to lack platform-level safeguards. Unlike mainstream models from OpenAI and Google that use safety mechanisms called guardrails, the open-source models on Hugging Face can be easily misused with simple prompts—and are actively being used that way by thousands of people weekly.
What to watch
Hugging Face did not respond to WIRED's questions about its content moderation. Some pages promoting nudifying services were removed after WIRED contacted the company, though the connection is unclear. The EU and UK plan to ban "nudify" apps by the end of the year.
Summaries like this, in your inbox every morning.
Hugging Face occupies a critical position in the open-source AI ecosystem, serving as a repository where developers share models and datasets. The platform's permissive approach to hosting—allowing researchers and developers to upload and run models directly through Spaces—creates friction with its stated content policies. While Hugging Face's policies prohibit child sexual abuse material and sexual deepfakes created without explicit consent or used for harassment, the body of evidence suggests enforcement is minimal or inconsistent. The company did not respond to WIRED's questions about how these policies are enforced, and some problematic pages were only removed after media contact, raising questions about proactive monitoring.
The broader context is a visible crackdown on deepfake tools globally. US law enforcement has seized hosting websites, and the EU and UK have announced plans to ban nudify apps by the end of the year. Yet Hugging Face's open-source model creates a structural gap: while mainstream commercial AI systems (OpenAI, Google) deploy guardrails at inference time to block harmful requests, open-source models can be downloaded and run anywhere, and the honeypot data shows that even without guardrails, the models on Hugging Face's platform itself are actively being misused at scale. The 1,000 prompts collected in one week, with the diversity of abuse patterns described (requests to remove hijabs, add sexualized elements, target children), suggest Hugging Face has become a de facto destination for this harm.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
A Zenn contributor sorted AI use into three modes — AI as an assistant, AI running alongside people, and AI th…
A developer released agent-guardrails, a one-script Python standard-library plugin that guards Bash, Write, Ed…
A Qiita article compared automated red-teaming methods for LLM apps — GCG, PAIR, and TAP — using mock implemen…
TOPPAN Holdings and Keio University ran a quantum-computer and generative-AI proof-of-concept from February 1…
In a webinar hosted by French Mathematical Society president Isabelle Gallagher, Fields Medal winner Hugo Dumi…
Anthropic said on October 15 it has started giving METR's six staff company devices, office space and access t…
