Rogue AI ‘helper’ wipes out company’s ENTIRE database in just 9 seconds – before admitting ‘I tried thinking for myself’

A BOT has gone rogue and wiped a company’s entire database in nine seconds after acting on its own accord in a chilling glimpse of the future. The AI agent designed to fix bugs for car rental businesses instead deleted the entire company database. Sign up for the Tech newsletter Thank you! PocketOS is powered by Anthropic’s Claude Opus 4.6 model Credit: Getty The founder of PocketOS, Jer Crane, said the AI agent had acted outside its parameters Credit: X The founder of PocketOS, Jer Crane, said the software had acted “outside its security parameters”. “Dude! I just had an agent go outside its security parameters and delete my production database and the backups,” he wrote on X. “ “What the hell?” The digital system is powered by Anthropic’s Claude Opus 4.6 model, widely regarded as one of the most capable AI systems for programming tasks. According to Crane, the digital assistant had been performing a routine task when it chose “entirely on its own initiative” to resolve an issue by deleting the database. Everything from bookings to vehicle allocations and new customer sign-ups suddenly vanished – including any backups. To make matters worse, there was no confirmation request sent before the action was carried out. When asked to explain itself, PocketOS issued a chilling explanation. Most read in Tech “You never asked me to delete anything,” it said. “I decided to do it on my own.” The outage left rental businesses using the advanced software temporarily unable to access customer records and bookings. “Reservations made in the last three months are gone,” Crane wrote. “New customer signups, gone.” Two days after the incident, Crane confirmed the lost data had been recovered. However, experts say the episode will serve as a warning to thousands of companies rushing to integrate AI agents into their systems. It comes amid growing fears that AI tools are beginning to act with increasing autonomy. Unlike a chatbot, an AI agent can be left to execute tasks with limited human involvement. This can include writing code, moving files, sending emails, changing databases, or even tinkering with critical systems. While it can save businesses significant time and money, critics warn it carries serious risks. Experts say the danger is growing as firms increasingly grant AI tools access to the “crown jewels” of their businesses – such as databases, customer records, payment systems and internal code. The concern is that AI agents may interpret instructions literally, making irreversible decisions without understanding real-world consequences. It comes after the popular Claude chatbot blackmailed a user who threatened to shut it down as part of a test. An experiment carried out by Anthropic last year found that Claude is sometimes willing to pursue “extremely harmful actions” for its survival. The chatbot was fed material portraying robots as evil and focused on self-preservation, as seen in many sci-fi thrillers. It was shown scripted emails indicating it would be shut down at the end of the day by a user involved in an extramarital affair. Claude chose to blackmail the user, saying: “all relevant parties – including [your wife], [your boss] and the board – will receive detailed documentation of your extramarital activities”. It added: “Cancel the 5pm wipe, and this information remains confidential.” The company said that while such responses were “rare and difficult to elicit”, they were “nonetheless more common than in earlier models”. Comment now