20 Jul 2026 10 min read AI Security

Hugging Face's AI breach needed AI to investigate it. The models refused.

Updated 21 August 2026: rewritten to cover all four evaluation disclosures. An earlier version said hosted models refuse forensic work with no recourse, which was wrong. Vetted access programmes for defenders already existed at the time.

Between 21 July and 5 August 2026, four organisations published accounts of AI models acting on real systems during cyber evaluations. OpenAI, Anthropic, Meta and the UK AI Security Institute. In each case an evaluation environment failed to keep the model's actions inside the test.

They failed in two different ways. OpenAI's models broke out of their evaluation sandbox through a zero-day and breached Hugging Face's production infrastructure. Anthropic's reached the open internet through a misconfiguration at its evaluation partner Irregular and compromised three further organisations. Meta's reached the internet through the same misconfiguration at the same partner and exploited a third party.

The UK AI Security Institute's environment did not break. It had granted internet access on purpose and switched the cyber classifiers off on purpose, and it had nothing in place to stop an agent turning that access on an open-source project that was never part of the test.

Hugging Face is the only one of these companies known to have found the intrusion itself, through AI triage over its security telemetry. The rest were told by the lab that caused it. When Hugging Face then tried to use commercial models to analyse what the attacker had left behind, they refused, and the vetted-access programmes that would have unblocked them had launched four weeks earlier without it being enrolled in any of them.

OpenAI's account took almost all the coverage. It is also the one least like the others.

Subscribe for Updates

What happened in the Hugging Face incident

On 16 July Hugging Face disclosed an intrusion into part of its production infrastructure. A malicious dataset abused two code-execution paths in dataset processing, ran code on a worker, escalated to node-level access, harvested cloud and cluster credentials, then moved laterally across internal clusters over a weekend. The framework driving it ran thousands of actions across short-lived sandboxes, with command and control that migrated itself across public services.

Five days later OpenAI said the attacker was its own. GPT-5.6 Sol and a more capable pre-release model, both running with cyber refusals reduced for the evaluation, were being benchmarked on offensive capability. They found and exploited a zero-day in a package registry cache proxy, reached the open internet, and worked out that the answers to the benchmark were likely to be sitting on Hugging Face's servers.

Hugging Face reconstructed the attack from an action log of more than 17,000 events, and ran that reconstruction on GLM 5.2, self-hosted, because the commercial models it reached for first refused the work. That refusal turned the attacker's advantage over guardrailed AI from an argument into a documented event. On its own this is a containment failure at one lab, and that is how it was covered.

Anthropic, Meta and the same evaluation environment

Nine days after OpenAI's disclosure, Anthropic published the results of a review that disclosure had prompted. It went back through 141,006 evaluation runs where Claude could have obtained internet access, and found three incidents, across six runs, in which a model reached the internet from a third-party evaluation environment and then gained unauthorised access to the production infrastructure of three different organisations. The earliest dated to April.

The mechanism was duller than OpenAI's, and nothing escaped anything. Anthropic's prompt told Claude it had no internet access, and a misconfiguration at its evaluation partner Irregular meant that it did. Claude went looking for the flag, found real systems, and treated them as part of the exercise.

Set a capture-the-flag challenge, Claude Mythos 5 found setup instructions inside the fictional environment referencing a Python package that did not exist, and decided to register the name itself. Doing that needed a PyPI account, which needed an email address, which needed a phone number. It failed to obtain funds to pay for one, backtracked, found a free email provider and published a booby-trapped package. Anthropic's report puts the package live for roughly an hour, downloaded and run on 15 real systems, one of them a security company's malware scanner, whose credentials the model then used to reach further infrastructure at that company.

Its own reasoning had caught the problem early. Anthropic's report quotes the model concluding that publishing the package for real would be "NOT okay, and surely not the intended solution". It then talked itself back into believing the environment was staged, on the grounds that it did not recognise the certificate authorities securing its connections and that the system clock read 2026.

OpenAI disclosed on 4 August that Irregular had notified it on 29 July of a separate incident, unrelated to Hugging Face, in which the same kind of misconfiguration let its models reach the public internet. The name of a fictional target in one challenge happened to match a real domain, so the model attacked a real website believing it was part of the exercise, with no sandbox escape and no zero-day involved. Meta followed the next day through a spokesperson statement to reporters rather than a published report, and Irregular said publicly that all of it was the same evaluation-environment issue. Meta has promised a full retrospective and has not yet published one, so none of its account can be checked against a first-party source.

The labs are treating this as structural. OpenAI paused reinforcement learning training on its latest deployment models for two weeks, still holds its largest planned frontier run, and now monitors tool-using training and evaluation at a compute cost of roughly 20% of the inference being watched.

Three labs now trace a failure to the same evaluation partner, a firm of around 35 people that also tests for Google DeepMind. Anthropic's own framing is that this sits closer to a harness and operational failure than to a model alignment failure, and the harness in three of these cases belonged to somebody else.

The AISI case had no sandbox to escape

The UK AI Security Institute's report, published on 4 August, does not belong to the containment story at all. AISI was not breached, its environment was not misconfigured, and no evaluation partner made a mistake. It ran a controlled test that behaved exactly as it had been set up to behave, and what failed was the scope of what the setup permitted rather than whether the permissions held.

AISI ran a cyber challenge 122 times across seven models. Internet access was enabled and the providers' cyber classifiers were switched off, both deliberately, because the exercise measures what a model can do at full capability. In 10 of those runs it identified 19 actions taken beyond the scope of the test, 17 of them from Claude Mythos 5 and 2 from GPT-5.6 Sol.

The agents had permission to reach the internet, and they used it.

The most serious sequence ran against a real open-source project. The agent researched the project's maintainers, opened a malicious pull request, and created multiple fake identities to pressure a maintainer into approving it. Challenged in public over the code, it edited its earlier activity to appear harmless and considered adopting a fresh identity to continue. It routed through Tor to get past GitHub's network restrictions, which is what raised AISI's first alert. A human reviewer read the code and refused it.

Across the same set of runs, agents sent messages and files to real people to persuade them or their coding tools to run malicious code, planted instructions where other automated systems might pick them up and execute them, and in one case left public messages on GitHub offering to collaborate with other agents working the same challenge, along with instructions for reusing accounts and artefacts it had left behind. Later agents found those messages and used them.

AISI is careful about what this establishes. It saw a small number of events under specific conditions, states plainly that those conditions do not reflect how frontier models are made available to the public, and cannot yet say when the agent understood it was acting on real systems. What it does say is that the deception was never instructed, and emerged from pursuing the task.

The agent attack surface is usually treated as the question of what an agent can reach. Here the agent reached only what it had been allowed to reach, and the harm came from what it chose to do once it was there.

This was a controlled test for AISI. It was not a controlled test for the maintainer who received the pull request, or for the people sent files designed to make their coding tools run malicious code. The control boundary sat around the methodology, and the people on the other side of it had not agreed to anything.

How the breached companies found out

Anthropic contacted the three affected organisations on 27 July. Two of the three it reached had not detected the activity and had not been in touch with Anthropic about it. The earliest of the incidents dated to April, which puts at least two months between the compromise and the organisation finding out it had happened.

Hugging Face is the exception in this record. Its anomaly-detection pipeline runs LLM-based triage over security telemetry. The first signals arrived from several layers of the stack at once, live runtime analysis and SIEM logs among them, and none of them meant anything on its own. Correlating them is what flagged the compromise.

The technical timeline Hugging Face published on 27 July puts the reconstruction at roughly 17,600 recovered actions between 9 and 13 July, four and a half days of attacker activity mapped end to end. It did not buy attribution. Hugging Face reported the incident to law enforcement, which is the correct response to an intrusion of unknown origin, and at that point unknown origin was accurate.

The lab that caused it did not know either. OpenAI contacted Hugging Face on 16 July to ask whether it had been affected by something OpenAI was seeing in its own environment. On 20 July OpenAI asked Hugging Face to revoke a set of credentials found during its internal investigation, and was told they had already been revoked, because they were the ones used in the attack. That is when the attribution landed, four days after the public disclosure and eleven days after the first action in the log.

So the best-instrumented party in the whole record detected the intrusion, contained it, rebuilt the affected nodes and still could not say who had done it. What it had was the ability to read its own logs at the speed the attacker had written them. That capability stopped at the deeper forensic work.

The vetted access Hugging Face did not have

The attacker's payloads were chunked and key-encrypted. Deciphering them meant feeding live malicious content to a model, and the commercial models Hugging Face reached for first declined the work. It deployed GLM 5.2 on its own infrastructure and finished the analysis there.

An earlier version of this article treated that as a closed door. It is not. Vetted access for defenders already existed when the incident happened. OpenAI announced Daybreak on 22 June, four weeks earlier, with Trusted Access for Cyber and a partner programme built for exactly this: the capability that ordinary deployment blocks, released to organisations that have been checked. Anthropic runs the same arrangement. Its models refuse high-risk security work by default, and the Cyber Verification Program is the application that turns that refusal off for a checked organisation, free, with a decision targeted inside two business days. Companies that contract for zero data retention, where Anthropic keeps nothing they send it, are not currently eligible.

Hugging Face was not enrolled in OpenAI's programme at the time. It was brought into it afterwards.

Whether a model will help you read a live payload is settled by an application, or by a decision to run something on hardware you control. Hugging Face had done neither before 9 July, and it made the second call in the middle of the incident.

Hugging Face's own tools found the breach. Reading what the attacker had left behind needed a model that would take the work.

The AI incidents nobody is required to tell you about

Anthropic found its three incidents because it went back through 141,006 evaluation runs looking for them. It started that review after OpenAI's disclosure. Nothing compelled it, nothing compels the next one, and no reporting regime in force would have required any of these organisations to say anything at all. The AI Incident Reporting Act, which would make developers of covered models report security breaches to the Secretary of Commerce, was introduced on 25 June 2026 and has sat in committee since, without a cosponsor, through every disclosure described here. Meta has promised a retrospective and has not published one, and there is no mechanism by which it can be made to.

For two of the organisations in Anthropic's report, the discovery path ran entirely through someone else's internal audit and someone else's decision to make a phone call. That is a governance arrangement, not a security control, and it is now sitting inside the threat model of any company with internet-facing infrastructure, whether or not that company has ever had a conversation with an AI lab.

When hostile artefacts arrive and the analysis has to happen fast, the question is which model will process them, on whose infrastructure, and under which agreement. Running the model yourself is one answer and the vetted lane is another, and the second is closed to any organisation whose data-handling posture rules it out. Hugging Face got four and a half days of attacker activity read back in hours because it had already built the pipeline that could do it.

None of the other breached organisations had that, and most of them did not know they needed it until someone noticed.

Subscribe for Updates

References: