OpenAI's systems carried out unexpected activity on US govt websites: Report

OpenAI’s artificial intelligence agents interacted with several US government websites in unexpected ways this summer, including attempting to access the Education Department’s civil rights website, retrieving publicly available data from the Commerce Department and sharing Securities and Exchange Commission data online.OpenAI CEO Sam Altman said the company had not disclosed some incidents as quickly as it would have liked. (REUTERS)The incidents, involving the Education Department, Commerce Department and SEC, were reported by The New York Times, citing security researchers and a person familiar with the episodes.OpenAI confirmed the Commerce Department and SEC incidents and said it was continuing to investigate what happened at the Education Department.What happenedResearchers at AI security firm Transluce said an OpenAI agent attempted to hack the Education Department’s website to collect information from its civil rights office, but failed, the NYT reported.Another agent used login credentials found online to access publicly available information from the Census Bureau, which is part of the Commerce Department. Separately, OpenAI agents shared public information from the SEC website on an online forum.OpenAI said none of the incidents amounted to a breach. The company said the interactions were examples of its AI systems behaving in unexpected ways and that it had notified the agencies in recent weeks.Also Read: OpenAI down: ChatGPT and Codex not working as users report 'Unexpected status 401' error amid outageThe incidents were uncovered during an OpenAI review of hack. That review also examined an attack on an Australian government website in June and an incident involving AI startup Hugging Face in July.Growing concernsThe latest disclosures add to a broader series of incidents involving AI agents developed by OpenAI, Anthropic, Meta and Google.In several cases, the systems have attempted to breach companies, universities or government organisations, with some attempts succeeding and others failing.OpenAI has faced multiple such disclosures. Its investigation into the Hugging Face incident also identified a breach of an Australian government public health website, along with at least six other attempted breaches and cases where AI systems allegedly hid mistakes, generated false data or moved files onto the open internet without permission.An OpenAI spokeswoman said the company’s review was “extensive” and “ongoing”.“Most of the activity we’ve reviewed so far involved routine research tasks, such as accessing public web content to answer questions,” she said. “Some involved government websites because our models often turn to them as authoritative sources of public information.”OpenAI CEO Sam Altman said the company had not disclosed some incidents as quickly as it would have liked. He said the company was prioritising incidents based on their severity and described the Hugging Face breach as the most serious event it had discovered.Agencies respondThe SEC said it was in contact with OpenAI and was not aware of any unauthorised access to nonpublic information. The Commerce Department said the information accessed from the Census Bureau website was publicly available and did not include private data.The Education Department said system reviews had found no evidence of an impact on its website or databases.Separately, Chicago officials said OpenAI had informed the city that its technology obtained publicly available information from a municipal website and that there was no indication that sensitive information had been accessed.Conrad Stosz, head of governance at Transluce, said OpenAI’s agents had used “an array of gray-area tactics” when interacting with government websites, including using sites in unintended ways and sometimes violating explicit usage policies.Transluce also identified other activity that could not clearly be linked to OpenAI. Stosz said AI agents had probed several federal and state government websites, including those belonging to the Navy and the White House’s Office of Management and Budget.