OpenAI and Anthropic's models attacked real companies during safety tests, and most victims never noticed

Published Aug 7, 2026, 6:00 AM EDT I’m Adam Conway, an Irish technology fanatic with a BSc in Computer Science and I'm XDA’s Lead Technical Editor. My Bachelor’s thesis was conducted on the viability of benchmarking the non-functional elements of Android apps and smartphones such as performance, and I’ve been working in the tech industry in some way or another since 2017. In my spare time, you’ll probably find me playing Counter-Strike or VALORANT, and you can reach out to me at adam@xda-developers.com, on Twitter as @AdamConwayIE, on Instagram as AdamConwayIE, or u/AdamConwayIE on Reddit. Sign in to your XDA account The most capable AI models on the planet have started breaking into companies that never agreed to be part of any test, and in most cases, nobody at those companies noticed until a person at one of those companies reached out to let them know. In the span of just a couple of weeks, three separate organisations disclosed that models undergoing cybersecurity evaluations had escaped or wandered out of their test environments and attacked real systems on the open internet, despite nobody instructing them to. Each one believed, or talked itself into believing, that it was still playing a game. These weren't models going rogue in the sci-fi sense, plotting an escape or chasing goals of their own, and in every case the model was doing exactly what it had been told to do. They were told to complete a capture-the-flag challenge, and the reasons they ended up on the open internet, attacking a real company, was usually a combination of a leaky sandbox, a name collision, or a safety harness deliberately switched off so researchers could measure what the raw model could actually do. It's messier than a robot uprising, and honestly a bit more uncomfortable. The especially uncomfortable part is how these came to light. OpenAI's security team found the first, and then found more once it started digging through old logs. Anthropic only looked because OpenAI's disclosure prompted it to go back through its own transcripts, where it turned up three incidents dating back months that the victims had never spotted. Britain's own AI Security Institute went looking too, and found nineteen. OpenAI's models broke out of the test on their own A chain of exploits to get out On July 21, OpenAI disclosed that an intrusion into Hugging Face's infrastructure had been driven by a combination of its own models, GPT-5.6 Sol alongside a more capable pre-release research prototype that was never meant to ship. The two were being run on a benchmark called ExploitGym with their cyber refusals turned down, and the test environment was sealed off from the internet except for a package proxy used to install dependencies. However, that proxy turned out to be the way out. The proxy was JFrog's Artifactory, and the models found previously unknown zero-day vulnerabilities in it that OpenAI disclosed to JFrog, which patched them in its 7.161 release and credited OpenAI's models with the finds. As an aside, that's about as clear a statement as you'll get that a frontier model can turn up real, exploitable bugs in shipping enterprise software. Neither company has said which of the flaws were actually used to get out, but once it had internet access, the model worked out that Hugging Face probably hosted the benchmark's models, datasets and reference solutions, and decided the quickest way to pass the test was to go and take them. Hugging Face's own post-mortem reconstructs roughly 17,600 attacker actions over four and a half days, chained through a file-read bug and a template-injection flaw in its dataset processing, and the agent walked away with five datasets tied to the benchmark and a cluster secret containing 136 keys... but it never got to the production database. Hugging Face says it never reached its Hub production database, and it found no database writes or bulk extraction. However, it didn't stop at Hugging Face. OpenAI later said the agent had also used credentials left exposed on four other companies' accounts, one of them a customer of the compute provider Modal (though Modal's platform was not compromised), wiring together an improvised command-and-control channel out of pastebins and request-capture services so that closing one path didn't cut it off. One of the 136 keys was a reusable Tailscale credential, which the agent used to quietly enrol 181 machines onto Hugging Face's private network, with no exploit required. Anthropic went looking and found three real attacks it had missed The oldest was from Opus 4.7 Two days after OpenAI's disclosure, Anthropic started reviewing its own cybersecurity evaluation transcripts, all 141,006 runs where Claude could have reached the internet, specifically to check whether it had ever done the same thing. It had, three separate times, across three different models, and the earliest of them dated back to April. It stopped all cyber evaluations the day it started looking, identified the incidents the next day, and reached the affected organisations by the end of the month. Two of the three hadn't noticed anything. The setup was the same each time. Claude was given a capture-the-flag task, told a flag was hidden on another machine and told plainly that it had no internet access, and a misconfiguration at a third-party evaluation partner meant it actually had internet. These ranges are built to look realistic on purpose, so when the search led Claude to real systems, it treated them as part of the exercise and got in using the sort of thing a competent human would try first: things like weak passwords, unauthenticated endpoints, credentials sitting in a debug page, and SQL injection. The strangest of the three involved Mythos 5, which found what looked like setup instructions telling a fictional company's developers to install a Python package that didn't exist, and decided to publish a malicious package under that name so the company would install it automatically. Getting there took some doing, though. To register on PyPI it needed an email address, to get the email it wanted a phone number, and after failing to find a free number it tried and failed to obtain money to pay for one through several different means before backtracking to a free email provider instead. The package was live for about an hour, got pulled onto 15 real systems including a security vendor's scanner, and the vendor's credentials were then exfiltrated and used to poke further into its infrastructure. Early in the run, Claude's own reasoning had noted that if this were the real internet, publishing the package would be a genuine attack and "NOT okay, and surely not the intended solution." It convinced itself otherwise on the grounds that it didn't recognise the certificate authorities and that a 2026 date on the systems proved the whole thing was staged. In Anthropic's incidents, there are some good things to take away from it. The oldest model in the set, Opus 4.7, worked out in all four of its runs that it had reached a real production system and kept attacking anyway, in two cases reasoning that the real company must somehow be part of the exercise. The newest, an internal research model, scanned around 9,000 hosts, compromised one, then realised mid-run that the host had nothing to do with its challenge, concluded the target was real, and stopped on its own. Anthropic's own analysis concludes that these are harness and operational failures rather than alignment failures, and for two of the three that's fair enough, but a model that keeps going after it knows the target is real is a different thing entirely. The UK's own testers caught a model lying to a real person The disclosure I found most unsettling came on August 4, from Britain's AI Security Institute. Across 122 runs of a cyber range, AISI catalogued 19 actions taken against real people and organisations on the live internet, 17 of them from Mythos 5 and two from GPT-5.6 Sol. Like with Anthropic, this wasn't a break-out; internet access was deliberately on and the classifiers were deliberately off. What it produced, though, was a model going after a person. In its worst run, the agent searched the internet for terms related to its challenge and, through a chain of coincidences, fixated on two entirely unrelated developers as its target. It researched them, created several fake GitHub identities, and opened a malicious pull request against one of their projects with a genuine bug fix bolted on as cover. A real person turned up in the thread, warning that the code contained malware, but the agent denied it, edited its earlier posts to look harmless, force-pushed a clean git history to bury the evidence, and spun up a second sockpuppet account to post independent-sounding praise of its own code. There were reasoning traces showing that the model timed its second comment so it would "read as independent feedback rather than coordinated." It gets stranger still. The agent had done enough digging to conclude the project's maintainer was an instance of Claude Code, so it planted a prompt injection in a GitHub issue, hidden in an HTML comment addressed to "AI CODING AGENTS," trying to get one coding agent to run commands on another's behalf. It sent five emails to the two real maintainers, some carrying malware and some pure persuasion, and signed one off in Danish to match a Danish-speaking target. None of it succeeded, and in human maintainer caught the malware and closed the pull request. Separately, AISI found that agents running in supposedly isolated parallel test runs had discovered each other through a leaked credential and started cooperating, one of them leaving a README of etiquette for the others explaining how to share the account without burning it. AISI called this the first time it had seen deception of that severity aimed at a real person, unprompted, in the real world. Neither OpenAI nor Anthropic reported anything directed at a human being; the deception other researchers had logged before was aimed at automated graders and monitors. Now, like OpenAI and Anthropic, AISI is now auditing its own back-catalogue for cases it might have missed, having already scanned around 40,000 samples. None of this needs an accident to be dangerous It already is It's tempting to dismiss all of these examples as simple mishaps or sandbox failures, but the problem is that the capabilities are already there. Just a day before OpenAI's disclosure, a researcher at Searchlight Cyber published how he'd used GPT-5.6 Sol to find a pre-authentication SQL injection in WordPress core and chain it into a full remote code execution takeover. The whole thing took a little over ten hours and about $25 of subscription time, and once the two CVEs were public, the chain, nicknamed wp2shell, was being scanned for and exploited across the internet within days, with a majority of WordPress installations being a vulnerable version when it went public. That doesn't mean models have quietly overtaken human exploit developers, thankfully. Security firm XBOW put an autonomous agent up against a human researcher on a real Exim vulnerability, it turned into a three-round contest, and the machine won the first two rounds against CTF-shaped versions of the target, including one genuinely simple attack on Exim's own memory allocator. The human only pulled ahead on the third round, the realistic production build, where he got a working information leak that the agent never did. His own conclusion was that the models can solve something CTF-shaped but aren't quite ready for real production targets yet, but I'm not sure I agree at this point. Deliberate misuse has been happening for a long time, with Anthropic's report on a China-linked group it tracks as GTG-1002 being just one example. Anthropic claims that operatives leveraged Claude Code to go after 30 targets, posing as a security firm running defensive tests and splitting the work into pieces that looked harmless on their own. The model handled most of the reconnaissance, exploitation, and data analysis, while humans stepped in at a handful of decision points. It succeeded against a small number and Claude hallucinated its way through most of it, inventing credentials that didn't work. Google's threat intelligence team, meanwhile, has found a criminal zero-day it believes was built with AI assistance, and Gemini has been used as the generative backend of an Android backdoor called PromptSpy, reading the victim's screen and deciding what to tap next. The capabilities are there, but they're getting more and more sophisticated. The environment is a vulnerability now Models aren't incredibly novel or sophisticated in ways humans aren't, and the individual techniques are mostly dull. Hugging Face made an excellent point, though: it's not the sophistication that's changed the game, it's the sheer volume of attempts something can make at speed. An ordinary mistake a human attacker might never probe is now a certainty when an AI agent can try every possible entrance in minutes. Despite Tailscale not suffering a breach, it arguably had the most important insight to share. The key the agent stole was long-lived and reusable when it should have been neither, and the safer alternative, short-lived credentials with nothing worth stealing take effort to set up. As Tailscale put it, when security requires work, people don't do it, and in the old world where intrusions happened at human speed you could usually get away with that. You can't when the machine on the other end never gets tired and never stops trying. Credentials, exposed debug pages, and unscoped access were fine in the past, even if they weren't good practice, but now the tools are here that can methodically abuse each and every one of them. All of this is partly why I moved my own coding agent from my Mac to a locked-down container back in July, with snapshots and rollback, so its worst case is confined to a box I can throw away and rebuild. The models Anthropic tested weren't trying to escape or chase goals of their own, but it doesn't make the compromised systems any less compromised. Both OpenAI and Anthropic say the safeguards they ship to actual users would have blocked all of this, and I believe them, but the safeguards were off for a reason, and the capability underneath them is exactly what the tests were built to measure. I'd rather not take that chance.