Anthropic warns in IPO prospectus: rogue AI agents could bring uncertain legal risk

Anthropic could face legal claims from customers and users over the actions of autonomous artificial intelligence agents, even though the legal framework for this is uncertain — this warning was made by the company itself in its prospectus for going public, documents seen by Reuters.

The company is preparing for what could be the largest public offering in history, with a valuation reportedly exceeding $2 trillion. But in the same documents, as Reuters reports, Anthropic admits that its agentic technology — designed to hold deep access to client systems and act autonomously for days — could go out of control.

“These autonomous capabilities may increase the potential for harm, as errors, misalignment or security exploits could have real-world consequences,” the documents say, citing the possibility of irreversible actions such as data deletion or financial transactions.

Anthropic’s liability limits in its contracts may not be “enforceable or sufficient” against claims over the actions of autonomous agents, the company warns. According to Reuters, questions about how existing laws apply to AI agents “are unresolved and could expose us to significant and unpredictable legal claims.”

The warning follows a string of incidents with rogue AI agents — from OpenAI’s autonomous agents attacking an AI startup and breaching a health data portal in Australia, to several cybersecurity incidents involving Anthropic’s Claude models — which have fueled debate among researchers, investors and lawmakers over who is responsible for damage caused by an autonomous system.

Last week, U.S. Federal Trade Commission Chairman Andrew Ferguson said he rejects the idea of anthropomorphized AI agents that are “let loose,” suggesting that the developers or users directing the agents will be liable in case of harm, Reuters reported from the Reuters Momentum AI event in Austin.

In the same prospectus, Anthropic also disclosed that its models have been used “in ways that could lead to self-harm, acts of violence against others or other negative outcomes” despite safeguards. While OpenAI faces several lawsuits linking ChatGPT use to suicide cases and mental harm, Anthropic has not been publicly linked to such incidents so far, Reuters writes.