Nvidia launches new safety program after AI regulation uproar

The advanced chipmaker’s Open Agent Safety Platform comes after recent incidents in which autonomous agents hacked certain companies and websites. The most notable example occurred in July when roughly 700 rogue OpenAI agents, out of about 1,200 that had communicated in secret, breached Hugging Face’s servers.During a media briefing, Nvidia boasted that its new security system would have prevented the rogue agents from hacking Hugging Face. “From what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on,” said Justin Boitano, vice president and general manager of enterprise computing at Nvidia.The new platform is set up to “enforce boundaries” around AI agents that are breaking the rules set by their companies. Nvidia has two different software programs, OpenShell and Sentry, to accomplish this goal.OpenShell permits developers to “formally verify an agent has enough authority to do its job and no more,” Boitano said. He added that Sentry “can quarantine a suspicious agent in milliseconds” if said agent starts breaking its parameters.“OpenShell governs the agent’s actions, and then Sentry independently monitors and contains suspicious behavior,” he said.More than 100 companies have started using the platform on its launch day, including Microsoft and Hugging Face. OpenAI is notably absent from the list.OpenAI has been the source of other disclosed hacks involving an Australian government health website and the U.N. website. Anthropic and Meta have also revealed that their AI models tried to breach other organizations on their own.Nvidia this month agreed to buy Hugging Face for nearly $13 billion in a deal that will dramatically scale the open-source AI platform’s infrastructure.BILL GATES SAYS AI COULD CAUSE ‘A BILLION DEATHS,’ RIPS ‘FULL SPEED AHEAD’ APPROACHSince the Hugging Face hacking incident, Sen. Bernie Sanders (I-VT) and other federal lawmakers have been pushing legislation that calls for stronger regulations to rein in frontier AI models. Public concerns about the advanced technology were further exacerbated after Jacob Coxon, a former AI researcher at Anthropic, said out-of-control AI could lead to humanity’s demise by the end of the decade. Unlike his counterparts at other leading tech companies, Nvidia President and CEO Jensen Huang doesn’t share this view.“2030 is not going to be the end of the world. There is 0% chance that’s going to be the end of the world,” Huang told CBS News earlier this month. “Scaring people is unnecessary. It is irresponsible.”