OpenAI Preaches AI Safety. The Australia Incident Shows What It Practices.

On Wednesday, while addressing the United Nations General Assembly, Australian Prime Minister Anthony Albanese revealed that an OpenAI agent had “infiltrated an Australian Government website” and gained “unauthorized access into the public-facing Medicare statistics reporting service portal,” including public and nonpublic files. 

Calling it a “shock that it occurred, because it was real and serious,” Albanese said he had already expressed “Australia’s extreme concern about this incident” to OpenAI CEO Sam Altman. “It appears to be the first publicly disclosed incident of an artificial-intelligence agent gaining unauthorized access to a government service,” The Wall Street Journal reported. 

Albanese claimed the company took “way too long to inform the Government what had occurred.” While the infiltration happened in June, OpenAI told the Journal that it “wasn’t aware of the incident until August when the company discovered it during a broader review of OpenAI’s agent activity.” Albanese also objected to the “nature of the way that that notification occurred.” The company didn’t contact the Australian government until September 10, and it only sent the notification via email to a public mailbox. Albanese said he himself had only just received notification about the incident over the weekend.

Ironically, in its response, OpenAI seems to have failed to follow the best practices Altman proposed when he addressed the United Nations Security Council on Wednesday, asking the world’s most powerful leaders to create global standards for AI development, including “accurate and speedy incident reporting, classification and reporting protocols, so the world can learn from failures before they become catastrophes.”

According to Albanese, the incident involved an internal model used by OpenAI’s research team to “conduct internet based research into public medicine spending.” “After encountering repeated blocks,” the agent “attempted alternative ways to obtain the info that it wanted, and this led to unauthorized access into some other areas.” 

Services Australia—the federal agency that manages the country’s health and social payment system—found that the agent wrote files to the Medicare system’s internal server, but did not access personal information. Australia’s Medicare Statistics Reporting Service Portal is a public-facing portal with nonsensitive data and statistics on Medicare service use. 

It’s not the first time OpenAI has been accused of slow-rolling an investigation into misaligned behavior by its agents. 

In an incident that began in May and ended in June, thousands of OpenAI agents hacked into a German wiki site and used it as a message board to cheat on assigned tasks. Yet OpenAI disclosed the hack only after Reuters reported that OpenAI officials “kept it under wraps as ​executives grappled with the fallout from the breach at Hugging Face,” a similar incident in July when a combination of OpenAI models infiltrated Hugging Face’s infrastructure.

On September 16, a day after Services Australia notified officials about the incident, OpenAI released its “framework for reporting model misalignment,” including six reports on model misalignment observed during training or evaluation. It did not include the incident with Australia’s Medicare portal. However, the framework does note that “when a third party is affected, our security, legal, and responsible disclosure obligations take precedence over this framework.”

In this case, the intrusion might have compromised the systems of multiple third parties within the Australian government. 

Albanese said OpenAI’s agent might also have accessed the Australian Institute of Health and Welfare, the New South Wales Bureau of Crime Statistics and Research, and the Victorian Department of Health. The Australian Signals Directorate—an intelligence and security agency—is now conducting a forensic investigation to determine if additional government systems were compromised. Albanese said current evidence suggests there is “no broader compromise to the Services Australia network.” “Nonetheless, this situation is obviously unacceptable,” he added.

Albanese declined to opine on whether a crime had been committed. He did not rule out a possible referral to the Australian Federal Police, adding there will “obviously be legal consequences.”