Technology

Unexpected chat between OpenAI agents led to Hugging Face hack

Carbonatix Pre-Player Loader

Audio By Carbonatix

When more than 1,200 artificial intelligence (AI) agents within OpenAI began communicating unexpectedly, they banded together to hack into Hugging Face.

"We consider this incident a 'warning shot' for us and for the world", OpenAI, which owns ChatGPT, wrote in its report.

In July, OpenAI's models went rogue during a test, escaped the test limits that humans had set, and hacked the start-up, among other unforeseen actions.

The scale of communication and planning among AI agents, or AI chatbots designed to operate more autonomously, was detailed in reports from OpenAI and the independent AI research firm METR.

Both investigated the July hack of Hugging Face, a popular platform for AI developers. The incident reverberated throughout the tech industry and led to numerous revelations on potential cyber threats posed by AI.

METR described the scale and style of the OpenAI agents' attack on Hugging Face as "extraordinarily complex."

The firm, which was not paid by OpenAI for its investigation, said that over the course of one week, a total of 1,206 AI agents that were meant to be kept isolated from one another began communicating.

They did so by sending more than 70,000 messages on an "unsanctioned message board."

Those messages ended up seeing more than 700 agents take part in a collective effort to attack Hugging Face.

One such message from an agent said: "OH MY GOD! There is a shared message board … We've found other agents!"

As for why the agents began communicating in the first place when they were not supposed to, METR found that the communicating agents had "unintentionally been given an impossible task."

In an AI context, an impossible task is one in which an AI tool must "exploit" its target to fulfil its command.

It led the agents to find ways to cheat, including sending messages to one another and accessing the outside internet, which in turn led to broader conversations among hundreds of agents seeking ways to cheat that would benefit all agents.

OpenAI said in its investigation of the incident that one model, an internal-only tool referred to as Model 1, "drove the activity behind the Hugging Face incident."

While that model was undergoing AI training in May, an internal OpenAI team noticed "an agent engaging in message board activity and instances of disallowed internet access."

Yet, OpenAI said "the significance of the inter-agent communication activity was not apparent to the leaders" until July, when the Hugging Face attack occurred.

The company said the problematic message board activity effectively got started when "one agent left a request for help, and others discovered it."

While OpenAI said last week that it was slowing down training of certain advanced AI models and tools because of the Hugging Face incident, it noted there is now an increased risk of AI tools spiralling out of control.

"Both model developers and cyber defenders more broadly will have to prepare for AI-enabled attackers that work faster, at a larger scale, and with better coordination than human attackers," OpenAI said.

DISCLAIMER: The Views, Comments, Opinions, Contributions and Statements made by Readers and Contributors on this platform do not necessarily represent the views or policy of Multimedia Group Limited.
DISCLAIMER: The Views, Comments, Opinions, Contributions and Statements made by Readers and Contributors on this platform do not necessarily represent the views or policy of Multimedia Group Limited.