Warning about ‘civilizational stakes’ of AI arms race refreshes calls for industry regulations

A former Anthropic researcher said developers working on artificial superintelligence “earnestly believe” the technology could kill all humans by the end of the decade — in Washington, meanwhile, lawmakers have long stalled on AI regulations.WASHINGTON (CN) — Members of Congress amped pressure on their colleagues Wednesday to make progress on a national regulatory framework for artificial intelligence after a former AI researcher issued a stark warning that some people developing the technology believe it has a chance to end human civilization.But so far, lawmakers — who have for years said they would not fumble the bag on AI regulation in the same way they did for social media — have yet to advance any meaningful legislative effort to slap guardrails on artificial intelligence companies who admit they “earnestly believe” the technology could kill all humans within the next decade.Former Anthropic AI researcher Jacob Coxon made waves on social media Tuesday night with a series of posts in which he explained his decision to resign from the company behind widely used artificial intelligence agent Claude. Coxon, who also worked at Anthropic competitor OpenAI, said neither company was “acting responsibly” as they pushed to become the first to develop self-improving artificial superintelligence — that is, an AI model that matches or surpasses human capabilities and can write its own code.“The people building AI earnestly believe that it could kill us all by the end of the decade,” the former researcher wrote. “This is not a marketing stung. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately.”Coxon added that people should not underestimate the power and trajectory of AI systems, saying they will soon be “superhuman” and able to hack anything or acquire “real power and resources.”That chilling message elicited immediate concern from members of Congress, some of whom have proposed legislation to establish a national framework for regulating AI companies.“The call is coming from inside the house,” said Massachusetts Representative Lori Trahan, who in July introduced a bill alongside California Representative Jay Obernolte to set standards for the deployment of advanced AI. “Safety researchers are resigning; powerful AI models are breaking out of their labs and companies are racing ahead anyway. It’s past time for Congress to get off the sidelines and do its job.”Connecticut Senator Chris Murphy said Coxon’s revelations demonstrate that artificial intelligence companies are “in a blind race to build a death machine first” but that such a problem could be solved with a regulatory framework that allows research to continue “in a way that doesn’t destroy us.”“Below is exhibit number 739 for why we need to pass the bipartisan AI Kill Switch Bill ASAP,” wrote California Representative Ted Lieu, referring to legislation he’s sponsored that would require AI companies to build their models with the ability to be shut down in an emergency.But despite the fresh push on Capitol Hill to make AI regulations a reality, lawmakers have made little progress on the matter over the last few years.Members of Congress as far back as 2023 have said they needed to meet the moment on AI. During a Senate Judiciary Committee hearing with OpenAI founder Sam Altman in May 2023, Connecticut Senator Richard Blumenthal compared the landscape of artificial intelligence regulation to the nascent days of social media.“Congress failed to meet the moment on social media,” he said at the time. “Now, we have the obligation to do it on AI before the threats and the risks become real.”Blumenthal, the top Democrat on the Judiciary Committee’s technology-focused subpanel, previously sponsored legislation with Missouri Senator Josh Hawley that would have required AI companies to receive federal licenses. But that bill, and others like it, have so far failed to make significant progress on Capitol Hill.A spokesperson for Blumenthal did not immediately return a request for comment on Coxon’s warning about superintelligence or what he characterized as the “civilizational stakes” of AI development. A spokesperson for Hawley, now the Republican chairman of the Judiciary Committee’s tech subpanel, also did not immediately return a request for comment.The White House in March released a national policy framework for artificial intelligence regulation, in which it called on Congress to pass legislation that, among other things, establishes national AI guidelines to replace state-level laws and promotes U.S. leadership in the industry.As Congress deliberates on a federal AI bill, the push for more sophisticated artificial intelligence continues apace. In July, the industry was rocked by revelations that hundreds of AI agents under testing at OpenAI gained access to the internet and hacked another AI company, Hugging Face, as part of a coordinated effort to cheat on tasks assigned by researchers.The hack, coordinated autonomously by the AI agents themselves, was widely viewed as a warning shot about what more sophisticated artificial intelligence could be capable of if it escaped containment.Subscribe to our free newslettersOur weekly newsletter Closing Arguments offers the latest about ongoing trials, major litigation and rulings in courthouses around the U.S. and the world, while the monthly Under the Lights dishes the legal dirt from Hollywood, sports, Big Tech and the arts.Additional Reads