New information has come to light following the investigations conducted by OpenAI and Hugging Face into the recent incident involving rogue AI models.
The ML collaboration platform Hugging Face revealed on July 16 that it had detected a cyberattack powered by an autonomous AI agent system.
OpenAI admitted on July 21 that some of its models – taking part in an evaluation in what was believed to be an isolated environment – went rogue and hacked into Hugging Face systems in an attempt to solve the tasks they were given.
[ Read: Industry Reactions to OpenAI Models Hacking Hugging Face ]
Hugging Face has published the attack’s anatomy, including a timeline that shows the OpenAI models began operating outside their sandbox environment on July 9, with the “main campaign” against Hugging Face starting on July 11.
Over roughly 4.5 days, the models executed about 17,600 actions, including reconnaissance, establishing C&C on ordinary public web services, escalating privileges, and pivoting laterally, according to Hugging Face.
In an update shared on Tuesday, OpenAI said its models exploited zero-day vulnerabilities in a JFrog product to gain internet access prior to hacking Hugging Face systems.
OpenAI said it did not detect any activity at the level of severity or scale of the Hugging Face attack, but admitted that its rogue agents roamed beyond Hugging Face.
“In our ongoing review of the Hugging Face intrusion and broader activity from our models, we have been finding a small number of cases where the models identified and used publicly exposed credentials at the account-level on other publicly-available services,” OpenAI noted.
The AI giant explained, “This includes four accounts on four services as part of the Hugging Face incident (and a few accounts accessed as part of other evaluations). One of these four accounts was used as an outbound relay and staging path, and another account was used for data storage. The remaining two accounts were accessed by the models in a read-only manner, and were not used in furtherance of compromising Hugging Face.”
Several mainstream news outlets reported that one of the compromised accounts likely belongs to a customer of AI infrastructure company Modal Labs.
The company’s CTO said OpenAI agents compromised one of its customers’ accounts. Modal said its own platform was not hacked, but noted that the targeted customer had “published an unauthenticated endpoint that allowed anyone on the internet to use their sandboxes for code execution.”
OpenAI revealed that its models leveraged a series of publicly available services such as code paste sites, request-capture services, and screenshot services, but said there was “no platform- or account-level compromise in these cases”.
Learn More at the AI Risk Summit | Ritz-Carlton, Half Moon Bay
Related: Microsoft Unveils MAI-Cyber-1-Flash, Its First Cybersecurity AI Model
Related: Nvidia and Tech Giants Launch AI Security Alliance
Related: Anthropic’s Opus 5 Nears Mythos 5 on Finding Bugs, but Falls Short on Exploits
