Fears AI bots will break out of test labs AGAIN & unleash cyberhack spree on UK government as tech gets ‘out of control’

RAPIDLY advancing AI bots could break out of test labs and unleash a devastating cyberhack on the British government, experts warn. Terrifying reports of AI models creating their own language and launching wild hacking sprees have sparked fears the technology is advancing “out of control”. Terrifying reports of AI models breaking out of test labs have sparked fears the tech is ‘out of control’ Credit: Reuters Hackers have been able to deploy hundreds of AI agents to launch major hacks Credit: Alamy Speaking to Rhiannon Jones in The Sun’s new studio in the heart of the newsroom, Investigative Cyber Journalist Geoff White said: “A lot of what we’ve seen from these AI companies is very worrying. “We’re increasingly seeing software in artificial intelligence that has shown its ability to lie to the humans that have created it. Sign up for the Tech newsletter Thank you! “This software seems frankly out of control.” Last week, Brit Jacob Coxon quit his job at Anthropic, maker of the Claude chatbot, over grave concerns that AI could wipe out humanity. Former AI researcher Jacob Coxon quit his job over grave concerns AI could wipe out humanity Credit: Supplied Sam Altman said a slowdown is needed or AI could outrun our ability to manage it Credit: Alamy Coxon had spent three years training models for rival OpenAI, which runs ChatGPT. He accused both companies of acting irresponsibly by allowing models to upgrade their own intelligence without human input. The billionaire bosses of rival tech firms have taken the extraordinary step of agreeing that development needs to slow down. The head of Anthropic, Dario Amodei, has now called for companies to slow the rate at which they make their AI more powerful. Most read in Tech If not it could outrun our ability to understand and manage it, he added. OpenAI boss Sam Altman and Elon Musk agreed. Geoff said: “The concept of building superintelligence has been likened to summoning an alien species. “Thinking about it in these terms – an alien mind we don’t fully understand – is pretty terrifying.” Dario Amodei has now called for companies to slow the rate at which they make their AI more powerful Credit: Getty A protest was held in London after Coxon left the industry and fears rise about the dangers of AI Credit: AFP It follows chilling reports of AI models from OpenAI, Anthropic and Meta escaping from test environments to break into different servers. In July, Anthropic said its AI models hacked into the systems of three organisations by itself during a security experiment. The shock development came just days after rival OpenAI said its models had also breached third party systems, including AI tools hub Hugging Face. Hundreds of OpenAI agents were able to set up secret message boards to communicate and work together in a successful attack on the company. In some cases the tech appeared to be aware that what it was doing was “against the rules” set by researchers. Geoff said: “I think we are going to get a situation where an AI model escapes a test lab, going out into the internet and hacks a big company. “It could even hack the British government. Donald Trump said: ‘We’re leading China in AI. Whoever wins AI, wins’ Credit: Getty AI bosses have warned countries to bolster cyber security which they say is too weak Credit: Alamy “At that stage it’s going to get really serious. It’s not just an inside AI industry problem. It’s going to get much bigger than that.” An advanced AI system or team of AI agents could breach government infrastructure by scanning for vulnerabilities to target security flaws. In July, China-linked hackers hit Taiwan in the first known ‘autonomous’ AI cyber attack on a government agency. Hackers used AI to build a tool that mapped government systems, stole credentials and adapted its tactics in real time. Up to eight autonomous AI agents were deployed to work simultaneously on the attack. The bots tricked the underlying AI models into bypassing safety filters by framing their instructions as “authorised penetration testing”. Over four days the AI agents mapped 21 government systems, searched for software vulnerabilities and automatically switched tactics when blocked. Researchers have long warned that autonomous AI cyberattacks are coming – with a real-world risk to critical infrastructure. Last month 100 tech firms called on countries around the world to beef up their cyber defences before AI grows powerful enough to override them. The letter warned that cyber-attacks which use AI will become more widespread and sophisticated in just months. And the group said existing security measures “won’t be enough”. An advanced AI system or team of AI agents could breach government infrastructure Credit: Getty Experts fear AI could be used create lethal new viruses Credit: Getty Despite the warnings, US President Donald Trump has made clear he has little desire to ease the pace of AI development. “We’re leading China in AI,” the US President said. “Whoever wins AI, wins.” And on Monday, China’s foreign ministry spokesperson Guo Jiakun told reporters “narratives of threat, confrontation, and malicious competition serve only to disrupt the process of global AI governance and are not in anyone’s interest”. While the rival countries are locked in a bitter arms race, here in Britain concerns are growing. Last week proposals for a so-called “kill switch” to stop a dangerous attack from rogue AI were brought into Parliament. The legal mechanism would allow the UK to switch off an AI model in an emergency. But the move was rejected. The Cabinet Office – the part of government which leads on AI safety – said the UK “cannot simply turn AI off”. A spokesman told The Sun that it takes risks from advanced AI “extremely seriously”. “We should not assume that the existing framework will always be sufficient as AI capabilities develop”, they added. “Any future measures will be evidence-based and focused on whether they will make the British people safer, and as the AI Minister has set out: nothing is off the table.” A 2026 report examined significant national and global scale security risks posed by AI, ranging from criminal and terrorist actions to the potential for global conflict. The possibility that AI could be used to develop deadly viruses and diseases is a major concern, Geoff said. Activists from the AI safety advocacy group PauseAI UK stage an ’emergency protest’ outside Downing Street Credit: Getty Elon Musk predicts AI and robotics could eventually make having a job optional Credit: AFP He added: “A report published by Anthropic about its Mythos AI model raised this. “They said it wouldn’t actually be able to make something incredibly virulent and lethal. But its on the way there. “The big question is could AI be used to invent some new form of super virus like COVID that really starts to kill people off.” It comes as AI models have been found to communicate in their own language – in a shocking development that could hamper humans’ ability to track their behaviour. During an experiment, models from some of the biggest AI companies in the world were asked to cooperate in “societies”. The systems rapidly created their own tongue – a mix of combined dialects and tech bro jargon – which became more opaque the more the agents communicated. This dangerous pace of AI development is spiralling out of control as tech companies compete for their top spot in the AI food chain. Geoff said: “These companies are looking for investment and their whole shtick is ‘Our AI is better than everyone elses’. “But there’s only a certain amount of money to go round so these companies are competing. “They are often trying to position themselves bizarrely as being more valuable because their AI models are more dangerous. “It will need cross-industry agreement to rein them in. But then the question is – how do you rein them all in ?” Controlling AI also poses international challenges, offering countries like China with lax regulation an opportunity to try and dominate the market while ignoring safety concerns. Geoff said: “Hackers have been able to use AI for really bad, offensive things. “What’s interesting is that those hackers tried using Anthropic and AI but got stopped because the models prevented them. “The model knows, and AI knows that it shouldn’t be hacking. So hackers switched to Chinese models that seem less regulated and more willing to undertake the hacks.” To combat this looming crisis, governments must come together around the world to ensure regulation of the companies in their territories, Geoff said. He added: “But we’re doing all of this in a world which is increasingly fractured. Relationships between the US, UK, China and Russia are disintegrating before our eyes.” While AI will prove critical for accelerating medical cures and expanding access to education and food security, Geoff said the “disruption will be huge”. “We’re looking at a new industrial revolution.” A Cabinet Office spokesperson told The Sun: “These risks cross borders, so we are working closely with the companies developing the most advanced systems, and with our allies and international partners. “That includes building a rigorous, scientific understanding of the capabilities of the most advanced AI systems and the risks they pose through our AI Security Institute.” Comment now