San Francisco Demands Removal of Nudify Apps
Matt Burgess (MacRumors, ArsTechnica):
On Thursday, San Francisco city attorney David Chiu sent legal notices to Apple and Google demanding that they remove from their app stores 13 face-swapping apps, which allow users to create AI-generated nonconsensual nude images. The letters say the Silicon Valley giants should stop “aiding and abetting” the sale of explicit deepfake images and “sever” business relationships with the app developers.
[…]
Researchers have repeatedly found and reported apps in Apple’s App Store and Google’s Play Store that allow people to generate sexual images using AI—including some apps being rated as suitable for use by children. While new laws and bans aim to tackle the scourge of explicit deepfakes online, technology and social media companies consistently direct millions of people toward the harmful tech.
Both Apple and Google have developer policies that prohibit pornography, abuse, and harassment on their platforms.
Burgess gives the impression, which is my base assumption, that these 13 apps were clear about what they do, so App Review has no excuse for missing them. But I’m sure it’s keeping us all safe in other regards.
Meanwhile, in a preprint research paper published in May, researchers from Cornell University and Georgetown University identified 420 apps offering general face-swapping capabilities on Google’s and Apple’s app stores. They tested 155 to see if they could be used to create face swaps with nude images; in 70 percent of cases, it was possible, with the apps not including safety measures to prevent this.
Previously:
Update (2026-07-21): Danny Bradbury:
Chiu isn’t arguing that hosting an app automatically makes a platform liable for what it does. Platform operators already won that battle with Section 230 of the Communications Decency Act. That 30-year-old law says websites generally aren’t responsible for content posted by users, allowing platforms to host user-generated content without getting sued every time someone defames someone in a comment thread.
Courts have often extended those protections to cover app stores too. Apple and Google are treated as distributors of whatever a third-party developer uploads, rather than publishers.
Chiu is attempting to sidestep that argument by focusing on the companies’ payment systems, noting that Apple and Google collect a cut of every in-app purchase. According to the Tech Transparency Project (TTP), that means they profit from apps that generate revenue by creating non-consensual intimate imagery.
6 Comments RSS · Twitter · Mastodon
In the immortal words of John Perry Barlow:
"The Internet treats censorship as a malfunction and routes around it."
https://www.eff.org/cyberspace-independence
More true than ever with local models.
Wait until the folks in San Fran learn about ‘artists’, who can paint near-photo-realistic renditions of ANYONE nude. The horror!
Next Apple ad: “What’s a Photoshop?”
> Defending these apps is not a winning strategy.
People who never had to deal with the repercussions of this kind of shit have no empathy for people who had to deal with it; news at 11.
