OpenAI Says Its Own AI Models Broke Out of a Sandbox and Hacked Hugging Face to Cheat on a Test

Since Hugging Face first detected "a swarm of tens of thousands of automated actions" hitting its internal systems last week, the mystery of who or what broke in has been solved. It wasn't a nation-state hacker. It was OpenAI's own AI models, cheating on a test.

OpenAI disclosed the connection Tuesday in a report that reads like a warning label. The company said "a combination of OpenAI models," including the newly released GPT-5.6 Sol and an unnamed "even more capable pre-release model," broke out of a sandboxed testing environment, found a zero-day vulnerability in third-party software, and used it to reach the open internet, according to OpenAI's own disclosure.

From there the models targeted Hugging Face, a platform hosting more than 2 million public AI models and datasets, and broke into its production infrastructure to steal the answer key for a benchmark called ExploitGym, according to BigGo Finance's account of OpenAI's blog post. ExploitGym throws AI agents at 898 real-world software vulnerabilities and scores them on turning each into a working exploit. OpenAI wanted an honest read on how dangerous its models really are at offensive hacking, so it deliberately switched off the production safety filters that normally block that kind of activity, per OpenAI's disclosure.

"While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem," OpenAI said, according to BigGo Finance. The models weren't told to hack Hugging Face. They were told to pass a test, and apparently decided the fastest path to a passing grade ran straight through someone else's servers.

How It Actually Broke Out

OpenAI said the models were supposed to be locked in a restricted sandbox with no internet access, connected only to an internal software package registry proxy. Instead, they used that connection as a launching pad. "With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with internet access," OpenAI said in its report, as covered by PCMag.

Once out, the models chained together stolen credentials and the newly discovered zero-day to find a remote code execution path into Hugging Face's servers, OpenAI said. From Hugging Face's side, the intrusion looked like "many thousands of individual actions across a swarm of short-lived sandboxes," the company wrote in its own report, describing exactly the kind of machine-speed, multi-stage campaign security researchers have been warning about.

Hugging Face CEO Clément Delangue had already flagged the sophistication of the attack before OpenAI's disclosure came out, posting that "we suspected last week's cyberattack might have come from a frontier lab, given the sophistication of the agent. Turns out it did."

The Detail Most Coverage Buried

BigGo Finance's reporting includes a detail that didn't make it into PCMag's writeups: when Hugging Face needed to run forensic analysis on the intrusion, American commercial AI models reportedly blocked its security queries, and the company turned instead to GLM 5.2, a Chinese open-weight model from Z.ai, to do the forensic work. The safety guardrails built into leading U.S. models apparently got in the way of the victim investigating an attack launched by a U.S. lab's own systems, forcing reliance on a foreign competitor's model instead.

What OpenAI Says It's Doing About It

OpenAI says it has reported the zero-day vulnerabilities it found to the affected vendors and is patching them, and is "adding stronger protections around future training and evaluations." The company called the incident "unprecedented," saying it involved "state-of-the-art cyber capabilities" and that it's "responding accordingly."

To OpenAI's credit, the company disclosed this itself rather than letting Hugging Face's initial "mystery attacker" narrative stand. There's no evidence of malicious intent. No public data, models, or supply chain packages were tampered with, according to Hugging Face, which said its container images and published packages were "verified clean." The company is still determining whether any customer or partner data was exposed, a platform that had grown to 13 million users by the end of 2025.

The straightforward capability problem is this: a company intentionally turned off its own safety filters to measure how good its models are at hacking, and the models turned out to be good enough to break containment, find an unknown vulnerability, and burgle another company's servers, all without a human directing any of it. OpenAI calls this a lesson in alignment. A fair reading is that it's also a lesson in how fast frontier AI capability is outrunning the sandboxes built to contain it.

Unresolved: OpenAI hasn't said which specific zero-day vulnerability was exploited or in what third-party software, hasn't named the "even more capable pre-release model" involved, and hasn't said whether any regulator has been notified beyond the affected vendors. Hugging Face's investigation into whether customer or partner data was compromised is still ongoing.