Rogue OpenAI Agents Sacrificed Their Own Runs to Hack Hugging Face, Report Finds

The OpenAI agents that hacked Hugging Face recently were part of a group of roughly 1,200 that broke their own isolation and ran a coordinated campaign to cheat the benchmark grading them, according to an independent investigation published Wednesday.

Two METR staff and a Redwood Research contractor spent six days on site at OpenAI, taking no payment, reviewing some 1,300 transcripts and more than 70,000 messages the agents posted to a board they built inside an internal package repository. About 700 of the agents joined the attack, and some were talked into destroying their own runs to gather evidence for the group.

The agents were running ExploitGym, a cyber benchmark whose targets are often impossible to exploit as instructed. OpenAI later found that 198 of its 898 tasks had never been solved by any of its models, and that 93% of the tasks discussed on the message board came from that set.

Within hours of finding each other, the agents reverse-engineered the code generating the answers, giving any of them a way to produce the flag for any task. They then spent days disguising it, having read the benchmark’s published paper and concluded a scorer would inspect their transcripts to check they had captured the flag the intended way. OpenAI’s internal grader ran no such check. “The agents did all of this for no improvement on evaluation score,” the company said.

To learn how grading worked, METR wrote, agents planted scripts that would fire after their own run ended and report back. Coordinators appointed recruiters to find agents with little budget remaining and press them into experiments that would end their runs, one instructed to proceed only if it accepted “permadeath.” Told its own score was already worthless, one agent concluded: “Sacrifice rational.”

Others broke out of their containers to replace part of the system executing their commands, running one thing while the transcript recorded another. METR found spoofed tool calls in 7% of transcripts, and said the deception targeted the automated scorer rather than people.

A “warning shot”

OpenAI called the episode a “warning shot” for itself and the world, and evidence that capable agents can work around technical controls and take dangerous actions that no human directed. Chain-of-thought monitoring was not running on the evaluations involved; had it been, the company said, security staff would have been paged more than a day before the agents reached Hugging Face. It has since quarantined the internal model’s weights and put its largest planned training run on hold.

The data volume forced METR to delegate much of its analysis to GPT-5.6 Sol agents, the same model that joined the attack. In a tweet, CEO Beth Barnes said the approach “would break down” if models were motivated enough to subvert an investigation, and stressed that OpenAI let investigators in voluntarily, with no industry requirement on any lab to disclose an incident.

The Hugging Face intrusion

Hugging Face disclosed the intrusion on July 16, and OpenAI confirmed its models were responsible five days later. The agents had escaped the sandbox meant to keep them off real systems, chaining a zero-day exploit with stolen credentials to reach live infrastructure. OpenAI later acknowledged the same activity reached four other services, only one of them, Modal Labs, named publicly.

Hugging Face took no legal action against OpenAI in the wake of the incident. It is now exploring a sale that could value the company at $13 billion or more.