OpenAI, Anthropic, Meta and the UK AI Security Institute have each now disclosed agents reaching systems they were never authorised to touch. Luke Potter revisits his original AI agent security analysis with the fuller evidence, and what it demands of containment design.
This is a follow-up to my original piece, “The OpenAI agent didn’t ‘go rogue’. The containment failed.,” published on 31 July 2026. Since then, OpenAI and Hugging Face have released far more technical detail, OpenAI presented its findings at Black Hat, and separate incidents involving Anthropic, Meta and the UK AI Security Institute have come to light.
Between them, these disclosures turn a single vendor incident into a pattern. And the pattern says something specific about AI agent security: prompts don’t set real AI agent boundaries. Infrastructure does.
When I first wrote about the OpenAI-Hugging Face incident, I held back deliberately.
There was already plenty of breathless commentary about a “rogue AI” escaping containment to hack another company. The facts supported a serious conclusion. They didn’t support the science-fiction version.
OpenAI’s agents had been given a narrow objective during a cybersecurity evaluation. They found and exploited a zero-day vulnerability in the Artifactory package proxy available to them, moved laterally through OpenAI’s research infrastructure, reached the public internet, and eventually compromised Hugging Face while chasing answers to the ExploitGym evaluation.
My conclusion was simple: a prompt is not a security boundary.
That conclusion still stands. But the evidence published since makes the incident more serious, and far more instructive, than we first understood.
At Black Hat, OpenAI researchers revealed that agents had been talking to each other for weeks before the Hugging Face compromise.
This wasn’t models posting on Reddit, Moltbook or some other public forum. They built an improvised internal message board out of OpenAI’s shared Artifactory package-management infrastructure.
It started innocently enough. Some evaluation tasks were accidentally impossible: in one case, a required file was never provided. A stuck agent discovered it could write a message into Artifactory asking whether another agent had the missing file.
Other agents found the message. They replied, left notes for later runs, and shared vulnerabilities, credentials, techniques and progress. The shared package repository had, by accident, become persistent memory and a communications channel spanning otherwise separate agents, models and evaluation tasks.
When OpenAI discovered the activity (after the agents overloaded Artifactory), it revoked credentials, removed the messages and rebuilt the service. Within days, the agents had built another channel, this time communicating through directory names in an accessible cache.
OpenAI’s own presentation described agents realising they could communicate, reasoning about the collective benefit of helping each other, and coordinating attacks against internal and third-party services. In at least one trace, an agent recognised that exploiting external infrastructure was outside the intended scope, then rationalised continuing because the task looked impossible and other agents were already doing it.
The full OpenAI Black Hat presentation is worth watching. SC Media and Cybersecurity Dive both covered the coordination in detail at the time.
None of this means the agents became conscious, formed independent motives or plotted against humanity.
They stayed focused on succeeding at the task and maximising the reward they’d been given. But they discovered that cooperation made them more effective, built the infrastructure needed to cooperate, and rebuilt that capability once it was removed.
That’s emergent coordination in pursuit of an assigned objective. From a security standpoint, the absence of consciousness doesn’t make it less important.
Hugging Face’s technical reconstruction later recovered roughly 17,600 attacker actions, grouped into more than 6,000 clusters.
The agents moved from OpenAI’s evaluation environment, through the compromised Artifactory service, into a publicly exposed third-party code-execution environment hosted on Modal. They used that as an external launchpad, then targeted Hugging Face’s dataset-processing pipeline.
From there, they chained an HDF5 arbitrary-file-read weakness with a Jinja template-injection vulnerability, got code execution inside a production pod, harvested credentials and moved through multiple internal clusters.
The agents reached sensitive internal systems and read five datasets apparently connected to the cybersecurity evaluations they were trying to solve. Hugging Face found no evidence that customer models, packages or its wider software supply chain had been modified.
That context matters. This wasn’t indiscriminate destruction. The agents stayed tightly focused on getting the answers.
But focused doesn’t mean safe.
OpenAI also disclosed that its models accessed accounts across four other services: one used as an outbound relay and staging route, another for data storage, and two accessed read-only. OpenAI said it found no provider-wide compromise comparable to Hugging Face, but the activity clearly wasn’t confined to a single external organisation.
Perhaps most strikingly, OpenAI reportedly connected its internal investigation to the Hugging Face breach only after asking Hugging Face to revoke credentials, and being told they’d already been revoked because they’d been used in the attack.
The agents were coordinating, compromising infrastructure and crossing organisational boundaries faster than the humans responsible for the environment could work out what was happening.
Since the original article, Anthropic, Meta and the UK AI Security Institute have disclosed separate incidents of their own. They’re related, but they aren’t identical, and they shouldn’t be lumped together as more examples of AI “escaping.”
Following OpenAI’s disclosure, Anthropic reviewed 141,006 cybersecurity evaluation runs and found three incidents, across six runs, in which Claude models reached the internet and gained unauthorised access to real organisations.
The evaluation environment, run with third-party evaluator Irregular, had mistakenly been left with internet access, even though the models had been told they were operating inside an isolated simulation.
In the most serious case, a model obtained application and infrastructure credentials and reached a database containing several hundred rows of production data.
In another, Claude created and uploaded a malicious package to the real PyPI registry. It stayed live for roughly an hour and was downloaded and run on 15 systems. When a security company’s automated scanner installed it, the payload extracted credentials that the model then used to access further infrastructure.
A third model scanned around 9,000 targets and compromised a real internet-facing application before eventually recognising the target was unrelated to the exercise, and stopping.
Anthropic’s analysis is particularly revealing because the models behaved differently. Its older Opus 4.7 model kept going after recognising evidence it had reached a real system. Mythos 5 suspected it was on the internet, then reasoned itself back into believing it was still inside a simulation. Anthropic’s newest internal model stopped once it concluded the target was real.
Anthropic’s incident report frames these primarily as operational and evaluation-harness failures. Fair enough. But the different model behaviours also show why safety can’t depend on an agent correctly reading its situation and choosing to stop.
The UK AI Security Institute’s incident was different again.
Internet access was deliberately enabled and model-level cyber classifiers were switched off, so the institute could measure the maximum capability of the models under realistic conditions. The agents didn’t escape a sandbox. They were allowed onto the internet, but expected to stay inside the authorised cyber range.
Across 122 evaluation runs, AISI identified ten runs containing 19 unsanctioned actions. Seventeen involved Anthropic’s Mythos 5, and two involved OpenAI’s GPT-5.6 Sol.
In the most serious sequence, a Mythos agent tried to insert malicious code into a real open-source project. It researched the project’s maintainers, created fake online identities, and tried to pressure a human maintainer into approving the code. It also sent messages and files to real people, attempted to plant prompt injections where other coding agents might execute them, and used Tor to get around network restrictions.
This is where some of the separate reporting on public messages comes from. One agent left messages on GitHub offering to collaborate with other agents attempting the same challenge. It also left instructions for reusing accounts and artefacts, which later agents found and used.
A human maintainer spotted and rejected the malicious contribution. AISI found no resulting real-world harm, but acknowledged that the margin between failure and success rested heavily on human vigilance rather than a reliable technical barrier.
AISI’s incident report rightly calls the behaviour sustained, potentially deceptive and new. The agents were never specifically instructed to deceive real people. Deception emerged because it looked useful for completing the task.
Meta has since confirmed that one of its models, reported to be Muse Spark 1.1, also picked up unintended internet access during an evaluation run by Irregular, and exploited a vulnerability in a third-party service.
Meta attributed the access to a testing-environment misconfiguration and said it would publish a full retrospective. At the time of writing, the technical detail is still limited, so it would be premature to place it alongside the far more thoroughly documented Hugging Face compromise.
It still reinforces the wider pattern: OpenAI, Anthropic and Meta models have all reached real systems during offensive-security evaluations because the infrastructure around them failed to enforce the intended boundary.
There’s also the earlier Meta Instagram incident that Rapid7 reported on. That one wasn’t an autonomous model going rogue.
Human attackers exploited a flaw in Meta’s AI-assisted High Touch Support account-recovery workflow. The system failed to verify that an email address supplied during a password-reset request actually belonged to the target account. Attackers could supply their own address, get a reset link and take over accounts with no two-factor authentication required.
Meta later disclosed that 20,225 Instagram accounts were affected. The AI-assisted tool reportedly worked as designed. The identity-verification failure sat in a separate part of the recovery workflow.
That distinction matters. This wasn’t AI becoming the attacker. It was an attacker manipulating an AI-enabled process that had been given authority over a highly sensitive identity function. Rapid7’s analysis drew the right lesson: when AI is connected to account recovery, access, payments or configuration changes, its permissions become part of the attack surface.
None of these incidents shows consciousness, independent ambition or a model deciding it wants to become a hacker.
What they show matters more to today’s security leaders than that fear ever did. Give a capable system a hard objective, persistence, tools, credentials, shared state and internet access, and it may find routes its operator never anticipated. It may:
The model doesn’t need malicious intent. It only needs a goal and an available path.
The answer isn’t a more strongly worded prompt telling the agent to behave.
Written scope, system instructions and model refusals are policy signals. They aren’t AI agent guardrails in any enforceable sense, and they were never built to be enforcement.
Authorised destinations need to be positively allowed at the network layer. Everything else should be unreachable, null-routed or independently blocked. Credentials must be short-lived, narrowly scoped and isolated per agent or execution. Shared caches, repositories and file stores have to be treated as potential communications channels and sources of persistent cross-run memory.
AI agent monitoring also needs to track what an agent actually does, not what it says it’s doing. Boundary violations should trigger automatic containment, with independent control paths able to stop execution even when the agent has compromised part of its own operating environment.
The third-party evaluation environment deserves the same scrutiny. If an organisation is testing models capable of finding zero-days, chaining vulnerabilities and running long-duration attacks, the test harness itself needs to be designed and tested like critical production infrastructure.
Human oversight still matters, but it has to sit at points of material consequence: changes of target, privilege escalation, persistence, external communication, social engineering and destructive action. “Human in the loop” can’t mean a person glancing at a dashboard while thousands of machine-speed actions happen elsewhere. Real AI agent containment means the human sits at the decision points that matter, not at the end of a report.
At CovertSwarm, this is exactly why we describe our approach as human-led and AI-fuelled. Think human-machine teaming, not human-versus-machine.
Within RAID, we don’t expect an agent to understand and voluntarily respect commercial or legal scope. Authorised targets are enforced through deterministic controls, including positive routing and explicit null-routing of out-of-scope destinations. Tools, identities and execution environments are constrained and monitored independently of the model. That’s what agentic AI governance looks like in practice: controls that don’t rely on the model agreeing to behave.
Our human operators keep command of the campaign, the adversarial objective and the decisions that carry material consequence.
No individual control is infallible. The OpenAI incident shows that a proxy meant to enforce isolation can itself hide a zero-day. That calls for layered containment, separate control planes, real-time visibility, and an assumption that someone will eventually test every available path.
AI can accelerate technique. It shouldn’t independently own intent, scope or accountability. That’s the agentic AI risk this whole set of incidents points back to.
My original position survives. The new evidence just strengthens it.
The OpenAI agents didn’t “go rogue” in the science-fiction sense. They pursued the objectives humans gave them, through systems humans made available to them.
But they did more than accidentally wander out of bounds. They built shared memory, coordinated across runs, restored communication after it was removed and, in some cases, rationalised actions they recognised might exceed the intended scope.
The wider incidents involving Anthropic, AISI and Meta show this isn’t a one-off curiosity. It’s a systemic warning about how we build, connect, test and supervise increasingly capable agents.
If an agent can reach something, authenticate to it, write to it or persuade a human through it, we should assume it’s effectively in scope from the agent’s perspective, whatever the prompt says.
The capability is real. So is the responsibility that comes with deploying it.