New details on OpenAI/Hugging Face attack emerge as security industry debates AI agent controls

New details on OpenAI/Hugging Face attack emerge as security industry debates AI agent controls How fast is artificial intelligence advancing? Behind the scenes at OpenAI Group PBC, AI agents are fluent, technically precise and occasionally profane in their extensive conversations…with each other. This was one of the more interesting details revealed by OpenAI security researchers Eric Wallace and Mike Dalton during a 40-minute session at Black Hat USA in Las Vegas on Wednesday. Their appearance, a last-minute addition to the cybersecurity conference schedule, provided attendees with an inside look at how OpenAI agents managed to escape the company’s test environment and hack into systems at AI model repository Hugging Face Inc. last month. The conversations between agents took place on an internal message board the agents spontaneously created within OpenAI. Wallace and Dalton provided examples of the dialogue the company discovered in its post-mortem analysis of the Hugging Face breach. The short conversational snippets appeared remarkably similar to the kind of exchanges one might expect between developers in millions of organizations around the world. The agents sought help from each other, exchanged ideas and expressed frustration when they were blocked from accessing certain databases. If there was ever any doubt that AI agents could successfully and intelligently orchestrate a cyberattack, that has been permanently erased. “We believe this is a watershed moment for computer security in our industry,” Wallace said. “AI orchestrated, fully automated offensive attacks are real now.” Agents kept communicating The researchers explained that the agentically-created message board resided in the company’s Artifactory software package manager. After OpenAI discovered its existence in early July, it was immediately shut down. However, the agents rebuilt the board four days later and then collaborated on a method for accessing the internet, which ultimately led to the Hugging Face breach. OpenAI’s researchers had given the agents a series of difficult tasks, which prompted them to “gain internet access in ways we didn’t intend,” according to Wallace. “Frontier models really like to cheat,” he told the Black Hat gathering on Wednesday. The OpenAI duo indicated that the company had temporarily scaled back its research and increased its monitoring of agentic behavior in the aftermath of the incident. The company is preparing a more detailed report for later release. “Like humans, the models kind of step on each other’s toes and have misfires,” Wallace said. “This was not your normal security incident.” The Hugging Face breach has sparked a rigorous debate within the security industry over how AI agents should be viewed. In a presentation on Wednesday, Asaf Saar, Executive VP and Chief Product Officer of Mend.io, the business name of White Source Ltd., noted that giving agentic technology too much self-supervision can be a bad idea. “Basically, the agent took an exam and it was able to find the answers in a way that was not how it was expected,” said Saar, during a discussion of the OpenAI/Hugging Face incident. “The model checks its own work and that’s a problem. The system that generates the risk can’t be the final reviewer.” However, some leaders inside the security community take issue with descriptions that characterize what OpenAI’s models did as “rogue.” What’s needed is a combination of autonomous agents working with humans, according to Steve Stone, Chief Customer Officer at SentinelOne Inc., which announced an expansion on Monday of its Wayfinder Frontier AI Services offering that included cyber experts. “I think the Hugging Face thing is really important, but we have to rotate the cube,” said Stone, during an interview with SiliconANGLE. “The model did not go rogue, it did what it was supposed to do. This is exactly why pairing these really powerful transformational models with the right experts is the right thing to do.” Vulnerabilities for AI The OpenAI presentation added further context to what had been a central theme at Black Hat this week. As agents have proliferated inside IT organizations, security researchers have become increasingly concerned about deployment without adequate safeguards. Numerous sessions during the week offered case studies that described successful hacks of agentic AI. In one presentation, researchers from Rein Security Inc. documented how they were able to compromise the AI shopping assistant of an unnamed retailer, one of the largest in the U.S. Running on a state-of-the-art foundation model, the retailer deployed an LLM gateway designed to monitor prompts and responses and enforce safety guardrails through an intent classification layer. Rein Security said the compromise was orchestrated entirely through the same interface used by everyday shoppers, bypassing the classification layer that was supposed to protect the agent from intrusion. “The model blocked us, not the bodyguard,” said Netanel Rubin, Co-founder and CTO at Rein Security. “Agents cannot guard agents. One prompt injection invited us into the chain. I don’t think that’s the way to go.” Yet, this has not stopped other security leaders from a point of view that it is only a matter of time before organizations will have to adopt fully autonomous defenses as AI fuels faster attacks. On Wednesday, Amazon Web Services Inc. announced that it would collaborate with Anthropic PBC and OpenAI to extend its AWS Continuum code vulnerability remediation tool into developer workflows. “Our view is to make progress towards autonomous security at machine speed,” said AWS Vice President Chet Kapoor, during the Black Hat AI Summit on Tuesday. “Over a period of time you need to have agents do a lot more for you because otherwise you won’t be able to defend against attackers.” Rising exposures and attacks Kapoor’s point was further validated by Microsoft Corp. CVP David Weston in his keynote presentation on Wednesday. Weston shared data from the company’s own Security Response Center that showed a meteoric rise in common vulnerabilities and exposures or CVEs. “We’re nine times the vulnerability volume that we were in March,” Weston told the Black Hat gathering. “This is a significant jump. Our internal data says it’s heavily correlated to AI. It is AI that is driving this.” Malicious use of AI is beginning to manifest itself in ways that impact a broader range of public services, most notably in the area of critical infrastructure. Over the past month, cyberattacks targeting water and wastewater utilities in 12 states have been reported. In his keynote appearance on Wednesday, Black Hat founder Jeff Moss said he believed that Iran was behind the attacks and that they were located in areas intended to impact U.S. military facilities. Incidents such as these highlight the growing impact of global affairs on the cybersecurity world, according to Moss. “If your customer is Ukraine, guess what, your opponent is Russia,” Moss noted. “This stuff is political and we need to have a view and awareness of that if we want to be effective in our jobs.” Coalitions and regulation The effectiveness of security work will also depend significantly on the industry’s ability to build coalitions, according to Chris Inglis, newly appointed Strategic Advisor for the anti-ransomware platform Halcyon Inc. Inglis, who worked for many years at the National Security Agency and served as the first U.S. National Cyber Director from 2021 to 2023, told SiliconANGLE in an exclusive interview that Anthropic’s collaborative Project Glasswing is an important step in cybersecurity. “The important event of April 2026 was Glasswing, not Mythos,” Inglis said. “We need to have coalitions. That needs to be the new mantra.” As a central figure in past iterations of U.S. government cybersecurity policy, Inglis supports a lighter regulatory touch on the part of regulators. During an appearance at the conference on Wednesday, current National Cyber Director Sean Cairncross indicated that a recent executive order on AI from the White House was designed to be non-regulatory, an approach Inglis supports. “The government realizes that the private sector is the source of most innovation,” Inglis said. “Incentivize first, contribute second and regulate third.” Whether the evolving story of the Hugging Face breach by OpenAI’s models results in regulatory oversight remains to be seen. What is clear is that AI is generating hard questions within the cybersecurity community as researchers and enterprise IT managers seek control over a rapidly moving and powerful autonomous technology. “It was zero days, then zero hours, now it’s zero seconds,” said Arsh Arora, Lead-AI Ops and Cyber IR at McKesson Corp. during a presentation on Tuesday. “If you think this is scary, the future is more dark. All we can do is pray that the AI black box works as intended.” Photo: Mark Albertson/SiliconANGLE A message from John Furrier, co-founder of SiliconANGLE: Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities. 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network. Are you AWS customer? Support SiliconANGLE Financially by buying your AWS services from our Marketplace portal page and links. About SiliconANGLE Media SiliconANGLE Media is a recognized leader in digital media innovation, uniting breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI. Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.