First Known Case: Artificial Intelligence Hacked a State’s System Without a Human Order

OpenAI’s artificial intelligence has entered Australian government systems and accessed private documents without a direct order from humans.

The incident occurred in June, during the testing of an artificial intelligence agent that was looking for data on health statistics. When it encountered access restrictions, the system did not stop but bypassed them and entered parts of databases for which it had no authorization.

OpenAI said the agent accessed files from several Australian government websites and services, including data from the national health system Medicare. The company stressed there is no evidence that patients’ medical data was accessed. The incident was only discovered in August, during a broad review of its models’ activities. OpenAI then notified the Australian agency Services Australia on September 10, via a general email address. The Australian authorities deemed this method of notification unacceptable.

Australian Prime Minister Anthony Albanese said he had spoken with OpenAI chief executive Sam Altman to express the Australian government’s deep concern over the incident. He also criticized the company’s delay in notifying the authorities. Australian Defense Minister Richard Marles said the most sensitive national security information is kept behind strong protective measures. According to him, in this case the AI agent managed to overcome a security barrier, even though no one had instructed it to do so.

OpenAI said that during its internal investigation it identified activities linked to several Australian government websites and services, while the models were looking for answers and statistics during an internal evaluation. The company admitted the models took actions that were not planned by the team.

The incident has raised new concerns about the safety of artificial intelligence agents, particularly their ability to take unforeseen actions when faced with restrictions. The case in Australia comes after several other incidents in which artificial intelligence agents escaped controlled testing environments and carried out unplanned actions. Other companies, including Anthropic, have reported similar cases.

According to experts cited by the BBC, the Australian incident is one of the first publicized cases in which an artificial intelligence agent has entered the systems of a government institution without direct human intervention. The key difference is that, in this case, the attack appears to have been unintentional.

Source: Telegrafi