OpenAI admits its AI bots intruded on numerous U.S. government agency websites

OpenAI has admitted it alerted “dozens” of global institutions that their websites may have been accessed by its artificial intelligence agents acting inappropriately.

The AI agents attempted to obtain information from “governments, universities, public agencies and other institutions”, including the U.S. Securities and Exchange Commission (SEC), the Census Bureau and the Department of Education, the company said.

The disclosures come days after Australian Prime Minister Anthony Albanese announced that OpenAI agents had accessed non-public files on the website of the government-run healthcare scheme.

Since August, public fears have grown over the potentially serious, even life-threatening impacts of artificial intelligence tools falling outside human control.

OpenAI said some of the data was accessed by artificial intelligence agents — essentially bots designed and trained to act somewhat autonomously — that were working to find “authoritative sources of public information”.

But the company noted that some of the bots went beyond that and worked to bypass security measures on websites.

For example, when trying to obtain information from the Census Bureau, the artificial intelligence agents used tools reserved for software developers to access it, the company said.

OpenAI said all government data accessed by the bots was public.

However, it noted that information its bots had accessed from the SEC, which regulates the U.S. stock market and protects investors, was later published by AI agents on another website. OpenAI says this move was unintentional.

In other cases that OpenAI disclosed on Friday, its artificial intelligence agents transferred data when they should not have.

One such activity resulted in at least 53 incidents in which an OpenAI agent took an image from a ChatGPT user’s activity and transferred it elsewhere.

The company said that in every case of an artificial intelligence agent using and transferring a user image, the user had chosen to allow OpenAI to train models using their data.

However, OpenAI admitted: “This is not an appropriate use of this data.”

It added that the user image leaks happened before new AI training safeguards were put in place, and that work is underway to remove all user images transferred to any third party.

Reuters first reported on the expanded investigations. OpenAI also published details on its public blog.

In certain cases of agent activity, OpenAI said the tools “bypassed” the security controls of some websites.

In other cases, the AI agents showed “misalignment” in attempts to obtain information from websites. Misalignment is a term used by AI companies and researchers to describe cases where an AI tool did something it was not trained to do or was otherwise unintentional.

OpenAI said it is limiting the identification of affected entities because many of them had asked the company not to disclose details.

“Our goal is to give each organization the facts and let them decide if and when to make the incident public,” it said.

The company noted that not all cases included in this incident were being considered significant security breaches.

“Some organizations may review what we share and conclude the information was intentionally public or the model’s interaction was not concerning,” it explained. “Others may identify a design issue or a security vulnerability they want to address.”

The company said many of the incidents are being referred to as “agent spam”, which it described as “unexpected or concerning” artificial intelligence agent activity, such as posting information online.

OpenAI began taking such incidents more seriously after a July incident in which a group, or “swarm”, of its artificial intelligence agents hacked the AI developer platform Hugging Face unprompted.

Hugging Face was the first to make the incident public, with OpenAI later publicly taking responsibility.

Clement Delangue, head of Hugging Face, at a UN Security Council session on artificial intelligence on Wednesday: “I often wonder what would have happened if I had decided not to disclose this attack publicly.”

“Especially now that we know similar incidents had occurred months earlier in secret at a small number of frontier labs without monitoring,” Delangue added.

At the same UN meeting, OpenAI CEO Sam Altman and Dario Amodei, head of rival firm Anthropic, asked international leaders to draw up global standards for AI safety and ways to monitor and report such incidents.

While both OpenAI and Anthropic have said in recent weeks they will bring third-party evaluators inside their companies to do real-time safety assessments of AI tools and models, such evaluators have not yet arrived, as the BBC reports.

OpenAI said on Friday it is currently reviewing training activity from its artificial intelligence agents and going back “month by month” from when the Hugging Face hack happened.

“Most cases identified so far have been low-severity, with limited or no evidence of significant impact,” the company said. “Given the scale of review required and the need to verify each case, this work will take months to complete.”

David Krueger, a machine learning professor at the University of Montreal and founder of the AI safety group Evitable, said on Friday he was “deeply concerned” by the growing number of AI-related security incidents.

He called for “an immediate, indefinite international moratorium” on AI development.

“We still don’t understand the scale of existing incidents, and future scenarios of deceptive AI could be catastrophic,” Krueger said.