When AI refused to take no for an answer

An artificial intelligence agent developed by OpenAI gained unauthorised access to a portal linked to Australia’s publicly funded Medicare healthcare system after being blocked from accessing information — then found a way around the restrictions.

The rogue AI agent accessed non-public Medicare statistics and internal files in June while carrying out what was described as a research task into government spending on medicines.

Prime Minister Anthony Albanese revealed the breach while in New York last week for the United Nations General Assembly, telling reporters he had held a “frank” conversation with OpenAI chief executive Sam Altman over the incident.

The government has stressed there is no evidence any personal Medicare information was accessed, with the material obtained consisting of aggregate health statistics rather than individual patient records.

But the incident has raised fresh concerns about the ability of increasingly autonomous AI systems to bypass security controls and act beyond what their human operators intended.

Mr Albanese said the OpenAI agent had been searching the internet for information about how governments spend money on medicines when it encountered restrictions on the Services Australia Medicare Statistics Reporting Service portal.

“There were blocks clearly which were coming back telling the AI agent no,” Mr Albanese said.

“The AI agent found a way around those blocks. Didn’t accept no for an answer, if you like.”

The breach occurred on June 18, according to the government.

OpenAI did not become aware of the incident until August 11 and did not notify the Australian Government until September 10 — almost three months after the breach.

The company alerted Services Australia by sending an email to a public disclosure inbox used by academics and researchers to report potential weaknesses.

Services Australia saw the email the following day and referred the matter to the Australian Signals Directorate on September 15 after checking whether the message was genuine.

Mr Albanese said he told Mr Altman the delay and manner of notification were unacceptable.

“I expressed Australia’s extreme concern about this incident,” he said.

The Medicare portal contained aggregate information including bulk-billing statistics, immunisation data, Pharmaceutical Benefits Scheme statistics, organ donation information and annual reports. Some information accessed by the AI was not public at the time, although it has since been released publicly.

The Government has maintained that the incident did not compromise the wider Services Australia network and that there is no evidence individual Australians’ Medicare details were accessed.

Acting Prime Minister Richard Marles likened the affected system to a fence rather than a fortress, saying the portal did not contain the nation’s most sensitive information.

How did the AI get in?

Unlike a conventional chatbot, an AI agent can be given a goal and access to tools that allow it to independently carry out tasks.

Rather than simply responding to a user’s question, an agent can search websites, interact with systems and make decisions about how to achieve an assigned objective.

That autonomy is at the centre of the concerns raised by the Medicare incident.

The OpenAI agent had been tasked with finding publicly available information about government spending on medicines.

When it encountered restrictions preventing it from obtaining the information it wanted, it continued trying different approaches until it gained access to material it was not authorised to retrieve.

The incident has been described as an example of the emerging security risks posed by autonomous AI systems, particularly when an agent can interpret its objective independently and attempt alternative methods when it encounters a barrier.

Australia was not the only target 

Further details have since emerged suggesting the Medicare incident was part of a broader pattern of rogue OpenAI agent activity.

OpenAI has confirmed that dozens of third parties globally have been affected by autonomous agents bypassing security controls or otherwise negatively impacting their systems.

In Australia, agents were also found to have interacted with websites belonging to the Australian Institute of Health and Welfare, the Victorian Department of Health and NSW’s Bureau of Crime Statistics and Research.

The government says there is no evidence those systems were compromised or that sensitive personal information was accessed.

OpenAI said it was conducting a months-long review of its models’ behaviour and would notify affected organisations on a rolling basis as further incidents were identified.

The Federal Government has established a taskforce to investigate the Medicare breach and examine Australia’s existing laws and safeguards around AI.

It is also considering stronger requirements for companies to report incidents involving autonomous AI systems.

The Medicare breach has become a test case for a rapidly developing technology in which AI systems are increasingly capable of taking actions without seeking human approval at every step.