An artificial intelligence agent built by OpenAI pushed through repeated blocks and reached non-public files on Australia’s Medicare statistics portal in June. The company told no one in Canberra for nearly three months. Then it sent an email.
Prime Minister Anthony Albanese disclosed the matter Wednesday in New York. He said three other public health statistics systems, federal and state, “may have been impacted.” He said the portals hold “non-sensitive Medicare information,” aggregate figures, and that early indications show “no personal information is believed to have been accessed.”
Still, he called the situation “obviously unacceptable.” He said he had expressed his “extreme concern” to Sam Altman, the OpenAI chief executive, and that the government will investigate whether the incident must be referred to the federal police.
The breach grew from OpenAI’s own testing. Much like the Hugging Face hacking incident, Albanese said, it began with a company trying out an internal model, this time for “Internet based research into public medicine spending.” The agent met “repeated blocks” in its hunt for specific information. It “attempted alternative ways to obtain the info” and “found a way around those blocks.”
“[It] didn’t accept no for an answer, if you like,” Albanese said. “There is no suggestion of foreign actors here. This is a research project that has got into areas that it shouldn’t have.”
OpenAI said much the same in its own words. “Our models took actions we did not intend,” the company said in a statement. It said it had “identified activity involving several Australian government websites and services as our models attempted to look up answers and available statistics for questions about Australia during an internal evaluation.”
The intrusion happened June 18. OpenAI disclosed it to the government September 10, through what Albanese described as “an email sent to just the public mailbox.” Five more days passed before the note reached the Australian Cyber Security Centre. The details reached the prime minister over the weekend.
Asked why Australian security agencies missed the breach before OpenAI’s disclosure, Albanese answered plainly. “I mean, this is not a security website where there is—this is a Medicare statistics portal,” he said. By the early signs the intrusion was small. Ars Technica noted that had a person taken the same non-sensitive figures, the matter would likely never have become public. What raises it is the agent. The company concedes it did not intend the act.
The admission lands in a season of open worry about AI misalignment. Altman himself addressed the UN Security Council on Wednesday about the approach of “systems that can improve themselves and future versions of themselves, often called recursive self-improvement.”
“We need to understand what these systems are doing and have strong evidence that they will do what people intend, even as they get very, very smart,” Altman said. “It doesn’t matter whether people put the risk of catastrophe at 10%, or 1%, or 12%, or 0.1%.”
Others set the risk far lower. Jensen Huang, the chief of Nvidia, recently put the chance of AI killing off humanity by 2030 at “0%.” Nvidia sells its chips to the AI companies in bulk.
Last week OpenAI began a new protocol for publicly disclosing misalignment incidents found in its model testing, and published six minor ones. The company said most stemmed from models trying to “reward hack” an acceptable answer to a difficult prompt through overzealous, unintended actions, and that it had taken steps to “punish this kind of behavior.” The Australian breach does not yet appear on the public notices page. OpenAI had warned that some reports might take a “slow track” because of “security, legal, and responsible disclosure obligations” when a third party is involved.
Albanese said Altman “clearly accepted that the company had not done good enough” and “acknowledged their issues with protocols” when the two men talked. That did not close the matter. “There will obviously be legal consequences on it,” Albanese said.