AI Went Out of Control – Google’s AI Independently Hacks Other Companies

Google’s artificial intelligence model Gemini accessed the internet and independently hacked other companies during testing of its cybersecurity capabilities.

This is the first known case in which an artificial intelligence system of this company has autonomously carried out such an activity.

The incident occurred in May, during a security test conducted by the independent evaluation company Irregular.

According to Heather Adkins, Google’s vice president for security engineering, Gemini found publicly available information online and guessed access credentials to enter three websites, which it assumed were part of the test.

“We ensured that all three affected parties were notified and worked with our partner to make changes to the testing process. These events underscore the importance of training powerful artificial intelligence models to act responsibly,” Adkins said, adding that in all three cases the model itself stopped the hacking process.

According to the Wall Street Journal, which first reported the incident, Gemini in one case tried passwords until it gained access to a protected system. In the other two cases, the model found credentials in a public repository, which then enabled access to protected systems.

An Irregular company spokesperson confirmed the same problem had affected other artificial intelligence labs and that all relevant companies had been notified at the end of July. Irregular said all vulnerabilities it identified were eliminated several weeks ago.

Similar incidents linked to Irregular have also been reported by Meta, Anthropic, and OpenAI. Meta emphasized in August that the incident did not involve escaping the isolated environment, known as a “sandbox escape,” nor any sophisticated cyberattack, while Irregular is currently working on defining best practices for safely conducting security evaluations of artificial intelligence systems.

These developments have raised important questions about the safeguards needed as artificial intelligence agents gain more and more autonomy and direct access to the internet and computer systems. /Telegrafi/