Last week, OpenAI claimed that a group of its AI models had broken containment, successfully hacking into the systems of open source AI platform Hugging Face to cheat on a benchmark test.
In the wake of the announcement, two very distinct narratives have emerged surrounding OpenAI’s claims. Some say it was essentially a publicity stunt, with the company setting parameters for the test that pushed the models toward outrageous behavior. But others, including certain prominent researchers, warn that the hack should serve as a warning shot for an even more severe AI-enabled cybersecurity disaster that’ll inevitably take place as models become more sophisticated.
“This is the first time, to my knowledge, that an AI system has autonomously committed a crime,” said New York Times journalist Kevin Roose of the event. “If a human did to Hugging Face what OpenAI’s models did to Hugging Face, they would be charged with computer fraud, and potentially sent to prison or fined or prosecuted.”
Debate will surely continue to rage among wonks and skeptics. And new details aren’t exactly tamping out the sense of alarm: on Tuesday, OpenAI issued an update to its ongoing investigation, claiming the incident was worse than initially thought. In addition to hacking Hugging Face, the company now says, its models “used publicly exposed credentials at the account-level on other publicly available services,” totaling “four accounts on four services.”
“We’ll continue to notify service owners directly, and have not seen evidence of broader impact to these providers or other accounts on their services,” OpenAI wrote, without elaborating on which services were affected.
The news further raised alarm bells among some cybersecurity experts, highlighting ongoing concerns over the tech’s ability to evade protective measures. It’s a possibility that researchers have warned about for years, and the incident suggests that the threat is now turning from a possibility into a reality.
On the other hand, more skeptical experts have become suspicious about OpenAI’s hair-raising tale. After all, we’ve heard a strikingly similar story from its biggest competitor, Anthropic, mere months ago. Could OpenAI’s latest admission be a bid to build hype to drum up excitement and prove to investors that its latest AI models are just as much of a cybersecurity threat as Anthropic’s fabled Mythos?
Experts also point out that the Hugging Face hack could’ve easily been prevented, further adding credence to the theory that OpenAI was looking for attention from the public. As cloud security firm Edera co-founder Alex Zenla told Wired, the hack was largely a result of callousness on OpenAI’s part.
“People are YOLO-ing really hard,” he said. “It’s shocking how little people have really thought about a scenario like this.”
“I consider all AI and anything AI touches to be fully untrusted — which is fine, you just need to build against that,” Zenla added. “And this situation proves the point. The fact that OpenAI wasn’t more paranoid about this seems kind of reckless.”
“A simple analysis of the actual risk has an actual simple answer,” security and compliance consultant Davi Ottenheimer told Wired. “The OpenAI mistakes were dead simple.”
As Wired points out, simple protections like fully isolating AI services from the internet could’ve prevented the hack, which have been intimately familiar to researchers for decades now.
Considering OpenAI is nearing a $1 trillion valuation and should have all the resources in the world at its disposal, major lapses in security should have anybody start questioning the company’s narrative.
Yet there could be truth to both versions of the story. It’s entirely possible that there is legitimate cause for concern as AI models become more sophisticated at identifying cybersecurity vulnerabilities. We’ve already seen frontier models flagging thousands of software bugs, underlining their growing competence.
But OpenAI is also heavily invested in showcasing its models’ capabilities to the world as it tries to keep up with Anthropic. That leaves the possibility that the company could’ve coordinated with Hugging Face to orchestrate the hack — or at least given its AI models a strong push in the direction of controversy.
More on the hack: OpenAI Says a Group of Its Models Broke Out of Secure Containment and Hacked Another AI Company
