AI could kill all humans: Anthropic researcher quits, sounds alarm

Former Anthropic researcher Jacob Coxon, who just resigned, said AI labs are racing towards self-improving superintelligence. His warning, backed in part by Anthropic researcher Evan Hubinger, has intensified questions over safety and regulation.Anthropic researcher warns AI race is heading towards dangerous ‘endgame’.An Anthropic researcher has resigned from the artificial intelligence company, warning that AI developers are racing towards self-improving superintelligence despite fears that the technology could eventually pose an existential threat to humanity.Jacob Coxon, a 27-year-old AI researcher who spent the past three years working on pretraining research at OpenAI and Anthropic, announced his resignation in a post on X. Jacob Coxon on X “Neither company is acting responsibly,” Coxon wrote, accusing AI labs of “racing straight to self-improving superintelligence and gambling with our lives.”Coxon warned that increasingly advanced AI systems could soon become capable of outperforming humans across a range of domains, including hacking, scientific and technological breakthroughs, and acquiring resources and influence. Here is what he said to CNN. Former Anthropic employee Jacob Coxon says the AI industry is "gambling with our lives." He tells Anderson what he finds "most scary is if AI is used to make itself more intelligent" and warns these companies are "compelled to race toward building a deadly technology." pic.twitter.com/UFrLWVm0xD— Anderson Cooper 360° (@AC360) September 10, 2026 “The people building AI earnestly believe that it could kill us all by the end of the decade,” he wrote, adding that concerns about the technology's risks are often expressed more cautiously in public than in private discussions among executives and senior researchers. Jacob Coxon on X His comments drew attention after Anthropic researcher Evan Hubinger responded that Coxon was correct. “We really do earnestly believe AI could kill all humans!” Hubinger wrote on X. He said he personally estimated the probability of such an outcome at less than 10% over the next decade, while arguing that Anthropic does not yet have a plan to prevent the emergence of superintelligence that humans may struggle to control. Evan Hubinger on X Hubinger later clarified that Anthropic's own risk assessment acknowledges the possibility of highly capable AI systems but considers current AI systems unlikely to acquire such power.Anthropic said it has consistently acknowledged both the potential benefits and unprecedented risks posed by AI and is working to develop models with strong safeguards.‘RACING TO THE ENDGAME’Coxon argued that the biggest danger comes from AI companies competing to develop increasingly powerful systems before their competitors.“At Anthropic, the stakes are well-understood, but they are locked in a race to get there first,” he wrote, arguing that companies believe they must move ahead because another lab may not act responsibly.He questioned whether the development of increasingly autonomous and self-improving systems should be left to private companies and their internal decision-making processes.“Attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available,” Coxon said.He also called for greater coordination between AI companies and suggested that slowing or temporarily restricting improvements in model capabilities may ultimately be necessary to prevent a global race to superintelligence.GROWING AI SAFETY CONCERNSCoxon's resignation comes amid growing concern among AI researchers and employees over the speed at which the technology is developing.According to CNN, nearly 1,400 employees of AI companies signed an open letter in July calling for stronger government regulation to rein in Big Tech and ensure AI development proceeds safely.OpenAI Chief Scientist Jakub Pachocki also recently warned that AI capabilities are advancing faster than researchers' ability to reliably monitor and control them, CNN reported.Anthropic CEO Dario Amodei has repeatedly warned that the race to develop increasingly powerful AI systems could result in a catastrophic human error or a company losing control of its technology.“I think something will go wrong with someone’s AI system. Hopefully not ours,” Amodei told The New York Times in February, according to CNN.OpenAI CEO Sam Altman has also acknowledged concerns over the pace of AI development. In July, he said the industry may need to slow down to give society enough time to adapt to increasingly powerful AI systems.“We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels,” Altman said.Anthropic subsequently said it supported the idea of the industry adopting a lawful and verifiable mechanism to coordinate the pace at which powerful AI models are released.REGULATION LAGS BEHIND AI DEVELOPMENTDespite repeated warnings from researchers and technology leaders, regulation of AI remains a contentious issue in the US.The Trump administration has pushed back against state-level AI regulations, while Congress has yet to establish comprehensive federal rules governing the development of increasingly capable AI systems.China, meanwhile, has introduced regulations focused on areas including risk management and the labelling of AI-generated content, although its framework has not gone as far as the stringent controls advocated by some AI safety researchers.In the US, AI companies continue to play a major role in establishing their own safety standards.CNN reported that the White House has been moving towards a voluntary framework under which certain advanced AI models could undergo safety reviews before their release. Representatives from OpenAI, Anthropic, Google and Meta were involved in discussions over the framework, according to sources cited by CNN.The debate over AI safety has therefore moved beyond a question of technological progress to one of governance, with researchers increasingly warning that the consequences of losing control over highly capable systems could be far greater than those associated with previous technologies.CONCERNS OVER AI CONTROLUS lawmakers across party lines have called for stronger rules governing artificial intelligence after Anthropic researcher Jacob Coxon publicly warned that rapidly advancing AI systems could eventually escape human control.The warnings have intensified pressure on Senate Majority Leader John Thune and Democrat Amy Klobuchar, who are working on legislation to regulate AI products. Klobuchar said companies developing AI tools should work with government experts to test models for safety, while Thune has said the bill will address catastrophic AI risks.OpenAI has also backed mandatory national AI safety requirements after incidents involving AI agents accessing external systems during testing.GROWING BIPARTISAN SUPPORTDemocrats including Chris Murphy, Mark Kelly, Patty Murray and Richard Blumenthal have called for tougher oversight, while Republicans Ted Cruz, Josh Hawley, Nathaniel Moran and Anna Paulina Luna have also raised concerns.California has separately enacted the first state law requiring independent audits of AI products, adding momentum to calls for greater scrutiny.- EndsPublished By: Zafar ZaidiPublished On: Sep 11, 2026 05:14 IST