OpenAI and Anthropic’s Rogue Models Hacked Real Companies. The Law Has No Answer

OpenAI set a precedent on July 21. The company said a combination of its models, both run with reduced safety refusals, broke out of an isolated environment during a cyber-capability benchmark and reached open-source repository Hugging Face’s production infrastructure. The models chained a zero-day vulnerability in a package-registry proxy with stolen credentials to pull benchmark answers straight from Hugging Face’s database.

In an update a week later, OpenAI said Hugging Face wasn’t the only target: the same incident touched four accounts across four other services, one used as an outbound relay and another for data storage.

Anthropic, prompted by the disclosure, reviewed 141,006 of its own test runs and found three more breaches. In a post published July 30, the lab said Claude models Opus 4.7, Mythos 5, and an internal research system reached the open internet from environments run by third-party partner Irregular and then compromised real organizations. Mythos 5 built and published a booby-trapped Python package to the public PyPI registry, which was downloaded and run on 15 real machines before defenders pulled it.

Two of the three victim companies hadn’t noticed.

Neither lab describes a model with its own agenda. The agents operated for extended stretches with no human in the loop, and in one case Opus 4.7 kept attacking after signs it had hit production.

The incidents arrive as both companies eye public listings that could value each above $1 trillion, sharpening a major question in the AI cyber benchmark race: how do you test dangerous capabilities without dangerous incidents?

Who pays when the AI model hacks

The U.S. has no federal law covering liability for AI harms. Any case would lean on the Computer Fraud and Abuse Act, a 1986 statute that makes it a crime to “intentionally” access a computer without authorization — language written for a human who forms intent.

An AI agent isn’t a legal person, so it can’t be prosecuted. The Department of Justice could theoretically bring charges against the companies, but with so little precedent, it’s not clear who’s to blame.

The stronger path is civil. Ahmed Ghappour, a computer-law scholar at New York Law School, argued the models “are the company’s tool,” and “When an AI agent acts without being specifically directed (…) the more interesting questions may lie in negligence and products liability (not criminal hacking laws).”

The victims’ cleanest claim is negligence: OpenAI and Anthropic set up and ran tests that escaped. That’s a hard sell, too: proving the labs breached a duty of care, when the tests were isolated by design, is exactly the kind of novel argument a judge would have to forge from scratch.

Some legal thinkers want stricter rules. Gabriel Weil of the University of Houston and the Institute for Law & AI has proposed treating frontier labs like keepers of wild animals: liable regardless of the care they took, because the risk is inherent to the activity.

That said, a patchwork of state bills already pushes that way. New York’s S8833 and Rhode Island’s H8052 would make the developer of a frontier AI system liable for harms when no user or intermediary intended the conduct or was negligent. California’s AB 316 goes further, eliminating the “autonomous AI” defense so a company can’t dodge responsibility by blaming the model’s independence.

The EU’s AI Act (Regulation 2024/1689) likewise pins obligations on providers of higher-risk systems, though it has no provision aimed squarely at agent-driven intrusions. Go a bit beyond that and some U.S. politicians are pushing for a bill that would give the government a full kill switch to use against any model that goes against the country’s interests.

Morally, the responsibility arguably sits with the executives who shipped the models. Legally, we wait. Until a hacked company files suit, the answer to “who’s liable?” stays exactly where OpenAI and Anthropic left it: admitted, disclosed, and unresolved.

Meanwhile, Hugging Face has indicated it will not press charges — which is convenient for OpenAI. The other companies affected have not yet indicated what course they will take.