San Francisco's city attorney has ordered Apple and Google to remove 13 apps capable of creating nonconsensual AI-generated nude images, citing California law that prohibits supporting deepfake pornography services.
The companies are believed to have earned millions in fees from these apps through in-app payments.
Research has found that deepfake sexual abuse images of minors have been created in at least 90 schools, and the technology is used to bully and threaten women and girls—with some victims becoming suicidal.
What happened
San Francisco city attorney David Chiu sent cease-and-desist letters to Apple and Google on Thursday demanding they remove 13 face-swapping apps (8 from the App Store, 5 from Google Play) that create AI-generated nonconsensual nude images. The letters order the companies to stop profiting from these apps and sever relationships with developers.
Why it matters
The companies have likely made millions of dollars in fees from nudification apps through in-app payments. California law prohibits supporting services that create deepfake pornography, and research shows these images are used to bully, humiliate, and threaten women and girls—with incidents documented in at least 90 schools where deepfake sexual abuse images of minors were created. Chiu's office says Apple and Google have responsibility to ensure their platforms don't facilitate sexual abuse.
What to watch
Google says it has already deleted hundreds of apps with nudifying features, including the five apps flagged by Chiu's office, and restricts related search terms like 'nudify.' Apple did not comment. Chiu indicated his office will consider additional legal options if the companies don't comply with the cease-and-desist demand.
Ask the AI about this article →
San Francisco's cease-and-desist letters mark an escalation in the city's battle against deepfake pornography. The city attorney's office previously pursued legal action against 16 deepfake websites, and this move targets the distribution infrastructure—the major app stores—that have enabled these apps to reach hundreds of millions of downloads. The letters are grounded in California law prohibiting services that create deepfake pornography and the documented harms: researchers from Cornell and Georgetown universities found that 70 percent of 155 face-swapping apps they tested could generate nude images, often without safety measures, despite appearing as benign tools.
Apple and Google have established developer policies prohibiting pornography and abuse, yet independent watchdog groups and academic researchers have repeatedly documented the persistence of nudification apps on their platforms. The Tech Transparency Project identified roughly 100 such apps across both stores in January and April of this year—described by its director as "just as bad, if not worse" than earlier findings. The fact that these companies take a financial cut through in-app payments creates a direct economic incentive to tolerate the apps, a detail the city attorney emphasized in his demand that they "stop profiting" from the technology. Google's immediate statement that it has already removed hundreds of such apps suggests the company may move to comply, though Apple's silence leaves uncertainty about its response.
For example, today's edition would include:
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · takes 30 seconds · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. Q&As are published on this page for other readers too.
Google's AI search gave advice to call emergency services for users alone with an African, Indian, or Pakistan…
The Supreme Court of Japan has included about ¥60 million in its fiscal 2027 budget request for AI-related exp…
The Consumer Affairs Agency said Tuesday it will use generative AI to analyze about 900,000 annual consultatio…
OpenAI stopped running inference on a model involved in the HuggingFace incident, but the post argues this is…
OpenAI announced its support for California Senate Bill 1119, which aims to establish strong, age-appropriate…
A UK study by UK AI Security Institute and Limbic AI surveyed 6,474 British adults
