Hugging Face Users Easily Created and Shared Abusive AI Images, Study Finds

The creation and distribution of nonconsensual intimate imagery, altered with generative AI, is one of the biggest and most serious challenges of the AI age. No AI company or platform is handling it well enough. A report published Tuesday by AI Forensics, a European nonprofit tech watchdog, found that models hosted on the AI platform Hugging Face were enabling the creation and sharing of these abusive and illegal images. Hugging Face doesn’t make AI models; it hosts existing ones, and it’s one of the most popular platforms for accessing and ranking open-source AI models. The report found it’s enabling access to AI tools for creating nonconsensual sexual AI imagery by “helping users compare, benchmark and optimize them, making image-based sexual abuse easier to scale.” Researchers manually investigated popular Hugging Face Spaces, which are online spots where developers can share access to their demo apps. They found that seven of the top nine Spaces for image editing allowed people to easily undress or “nudify” people in images without adversarial attack methods. They say that 73% of users’ prompts in those Spaces are sexual, and the abusive images overwhelmingly target women. Hugging Face told CNET that after receiving a copy of the report before it was published, the company conducted the same testing as AI Forensics and took action against Spaces that were intentionally misrepresenting their purposes. It denies a “systemic lack of moderation” that the report accuses it of, saying that several “artifacts” were removed under its current moderation procedures before the report was published. “We’re always grateful for work that provides information on [nonconsensual intimate imagery] issues, but find misconceptions in this report make it harder to act upon,” a Hugging Face spokesperson said. Nonconsensual intimate imagery created with technology, sometimes colloquially referred to as deepfake porn, was a problem before AI. But the generative tech has superpowered the issue, making it easier, quicker and cheaper than ever for people to perpetrate harm. And it’s not the first time we’ve seen an AI platform enable this kind of abuse; X (formerly Twitter) is a haven for this content, letting people use Grok to create and share nonconsensual “undressed” images on the platform. Platforms like Hugging Face and AI developers like xAI, OpenAI and Google have a shared responsibility to prevent their tech from being used for evil. We’ve repeatedly seen over the past year that as AI image-generation and editing tools become more capable and widespread, AI-powered image-based abuse rises. Hugging Face was also recently hacked by an autonomous AI agent sent by an OpenAI model, after it escaped the virtual confines of its testing sandbox. In response, CEO Clem Delangue asked OpenAI to be fully transparent with the model so it can be studied by the public and also pay up – not in cash, in something far more valuable: compute.