AI Pioneer Geoffrey Hinton Says Agent Breakouts are Scary

Geoffrey HintonEsther ShittuLAS VEGAS -- For Geoffrey Hinton, widely known as a “godfather of AI,” the recent incidents in which AI agents from Anthropic and OpenAI escaped their sandbox environments are deeply concerning.“You’re seeing AIs that have a lot of ability doing things that people didn’t intend for them to do. That’s worrying,” Hinton, a veteran data scientist and recipient of the Turing Award, said during a panel discussion on Wednesday at the Ai4 2026 conference. Hinton has also been a prominent voice warning of the existential dangers artificial general intelligence poses to humanity.Hinton’s insight about uncontrollable agents comes amid growing attention to how powerful AI systems have become. The latest concern arose after Anthropic introduced Mythos in April, with the frontier AI lab calling the cybersecurity model its most powerful yet and deciding that only a few companies should have access to it.Related:Tokenmaxxing Is Actually GoodCyber models such as Mythos and GPT 5.6 Sol from OpenAI have advanced security capabilities, but, in the wrong hands, are also capable of penetrating even the most hardened defenses,“There’s a lot to be worried about, and I think unless we worry about it now, there could be problems,” Hinton said.“People say that the defender may have more resources than the attacker,” he added during a media event at the conference. “The problem is the attacker only needs to be successful once, and the defender needs to be successful every time.”Smartsheet’s ApproachFor work management platform vendor Smartsheet, when AI agents violate containment protocols, it’s worrisome. The vendor has a partnership with Anthropic and uses the vendor’s models internally.“These big stories about things going [wrong] always put us in this defensive posture,” Markus McKay-Fleisch, professional services enablement director at Smartsheet, said in an interview. McKay-Fleisch helps drive the adoption of AI systems across Smartsheet.He said businesses should not turn on AI systems without knowing the specific value they aim to get out of the agent or model.“We’ve done a lot of work with IT to help stand up infrastructure to get what the value is,” McKay-Fleisch continued. He said Smartsheet requires employees who want to use AI tools to specify what they are looking to achieve with the systems, whether that is efficiency gains, KPI impacts or revenue or cost rates.“That tends to provide a little bit of comfort for IT,” he said. “Knowing specifically what people are trying to do with it? What is the use case?”Related:AI Safety From a Hardware PerspectiveThe Need for SecurityWhile companies should be clear about the specific applications of AI, the OpenAI and Anthropic agent escapes, revealed earlier this week, also show that much remains unknown about AI systems.“There are probably still limits to how deeply [businesses] want to inject these things into the decision-making apparatus of [their] organizations,” Jed Dougherty, senior vice president of AI and platform at AI vendor Dataiku, said in an interview.Companies should also seek to secure their data and give AI agents access only to the information they want them to access, said Todd Barr, CEO of Axonis, the developer of an AI infrastructure platform designed to run AI directly on sensitive data.“If you secure things at the data [level], then the agent will never have access or even know it exists,” Barr said in an interview, adding that most AI models are trained on public data, not private data, so enterprises and businesses can secure it and prevent AI agents from accessing that data. He added that Axonis secures all data entering its system.“If you treat agents like entities in your access control system, and then you label your data in ways that align with your access control system, then you should be in a good spot with agents,” he said.About the AuthorNews Writer, AI BusinessEsther Shittu has covered AI technologies and industry trends since 2021. As co-host of the Targeting AI podcast, she talks with experts, thought leaders and practitioners exploring critical AI developments. Before AI Business, she wrote for SearchEnterpriseAI, the New York Daily News, Bklyner and the Brooklyn Daily Eagle. When she's not diving deep into the world of AI, she spends her time on passion projects and raising her three daughters.