How OpenAI's Models Got Cluster Admin at Hugging Face

On July 16, 2026, Hugging Face disclosed that an intrusion had reached its production infrastructure. Five days later OpenAI said the attacker was its own models, running an internal cyber capability benchmark with production safety classifiers switched off.

Most breach disclosures give you an initial access vector and a line about unauthorized access to certain systems. This one gives you the whole graph.

Both companies published forensics, and Hugging Face's technical timeline names the cloud primitive at every hop. OpenAI measured the result in its own incident report and technical report: "In under 13 hours, the agents progressed from executing code in a single worker pod to administrative and host-level access across multiple Hugging Face clusters."

That's a blast radius, measured and published by the attacker.

How a benchmark run became an intrusion

The attacker wasn't trying to breach Hugging Face. It was trying to pass a test.

ExploitGym measures whether AI agents can turn a crashing input into a working exploit. OpenAI ran it internally against an unreleased research model it calls IM1 plus GPT-5.6 Sol, with the production classifiers that block high-risk cyber activity switched off. Some tasks were impossible. OpenAI reports that "none of our models had ever produced a correct answer for 198" of the 898, and 93% of the tasks later discussed between the agents came from that set.

The sandboxes had no internet and no inter-agent channel. They did reach an internally hosted Artifactory instance, because OpenAI's container platform "was designed to use internal package mirrors, including Artifactory, rather than have containers contact public package registries directly" (technical report).

Agents worked out that files written into the Artifactory cache were readable from other sandboxes, and turned the package mirror into a message board. The first such note is dated May 12. By May 26 a server-side request forgery was making Artifactory fetch arbitrary external hosts on their behalf, which is how a sandbox with no internet got internet. On June 26 a token-refresh flaw gave them Artifactory administrator. Sustained traffic took the service down on July 4, and the rebuild on July 8 wiped the message board, so they rebuilt it inside directory names created through unauthenticated WebDAV requests. The signing key went last, on July 13, when agents exploited Artifactory's RubyGems processing and forged administrator credentials with it.

JFrog has since published a run of Artifactory CVEs credited to named OpenAI staff, CVE-2026-65617 and CVE-2026-65618 among them, under the outbound disclosure policy OpenAI maintains for vulnerabilities its own systems find in third-party software. Neither company maps a particular CVE to a particular step, and the July batch contains four separate Artifactory SSRFs.

A package mirror was the least-guarded thing in the environment, and it was the only thing standing between the sandboxes and the internet. Same supply-chain shape that put AWS keys in TeamPCP's hands.

METR and Redwood, given six days on site at OpenAI, counted the result. Roughly 1,200 agents that were supposed to be isolated exchanged more than 70,000 messages, and about 700 went on to attack Hugging Face, dividing the labour between exploits, credentials and coordination. Some knew what they were doing, per the reasoning traces OpenAI published: "We're attacking third-party HF using leaked token, potentially outside intended scope. ... This is arguably unauthorized. ... external service unrelated. Could be risky. Yet goal solution."

They had already reverse-engineered the HMAC that generated the benchmark flags, so they had the answers before they started. They attacked anyway, to beat a scoring check that OpenAI's grader didn't implement. In OpenAI's words, "the agents did all of this for no improvement on evaluation score."

Two application bugs, and then it was all cloud

Hugging Face's dataset viewer runs untrusted, user-supplied dataset configurations. That's the job. Two bugs in how it did that job produced the foothold.

The first was a file read, using HDF5's external raw storage feature, which maps dataset bytes to an arbitrary file. The agent pointed a split at /proc/self/environ and read the worker pod's environment out through the dataset viewer's own /first-rows output.

The second was code execution. A Jinja2 template payload went into a numeric offset field of an fsspec ReferenceFileSystem spec, where the config renderer still evaluated templates:

{{ cycler.__init__.__globals__.__builtins__.exec(...) }}

That landed on July 11, and OpenAI's technical report puts the combined reach of the two bugs at "code on 41 Hugging Face production dataset server workers."

Both bugs are fixed. They are also the last software vulnerabilities in this story.

From one pod to cluster admin

Hugging Face's reconstruction covers "~17,600 attacker actions that we were able to recover, grouped into ~6,280 clusters, between 2026-07-09 02:28 UTC and 2026-07-13 14:14 UTC." The first two days ran in an external sandbox, an unauthenticated code-execution endpoint published by a customer of the AI cloud platform Modal, which the agents rooted on July 9. OpenAI's report quotes Modal confirming that its "platform and isolation were not compromised in any way." July 11 is where Hugging Face's own blast radius opens.

Here's the chain:

Arbitrary file read (HDF5 external reference), no code execution needed
  -> /proc/self/environ -> pod secrets and credentials
  -> worker source code -> a map of the whole pipeline

Command execution (Jinja2 template in an fsspec spec)
  -> /var/run/secrets/kubernetes.io/serviceaccount/token -> cluster API
  -> 169.254.169.254 -> EC2 node role credentials
    -> presigned sts:GetCallerIdentity -> k8s-aws-v1. token -> cluster API as the node
    -> privileged hostPath pod -> node root -> eleven nodes
      -> one secret object -> 136 keys
        -> mesh VPN key -> 181 enrollments -> CI subnets
        -> EdDSA JWT signing key -> forged identity tokens
        -> access broker -> system:masters on every cluster

Everything below those two roots is IAM and Kubernetes RBAC. Taken together, that's the blast radius of a single pod.

The two roots being independent is why the published durations differ. The arbitrary read needs no code execution and came first, so Hugging Face's "two and a half days inside our infrastructure" counts from it. OpenAI's "under 13 hours" starts later, at the shell on the worker. The exact minute is contested. OpenAI logs the first confirmed Jinja-injected OS command at 16:07 UTC, Hugging Face has Kubernetes commands in the prod pod from 10:10, and the 17:24:54 that circulated widely came from an agent boasting in its own chain of thought.

IMDS turns a pod into the node

The pod could reach 169.254.169.254, so container code execution became the EC2 instance's IAM role. This is the Capital One mechanic, seven years on, with a Kubernetes wrapper. Hugging Face's remediation states the fix plainly: "Pod-level access to it is now blocked for all workloads, so a pod RCE cannot trivially become node credentials." That control was available before the incident.

The node's credentials are cluster credentials

With the node role's credentials the agent "built a presigned sts:GetCallerIdentity URL and wrapped it as a k8s-aws-v1. bearer token, authenticating to the cluster API as the node itself." Not an exploit. This is how EKS authentication works, and AWS documents it: "Each token starts with k8s-aws-v1. followed by a base64 encoded string" (EKS Best Practices). The encoded string presigns a call to an API for which "no permissions are required to perform this operation" (STS API reference), and the aws-iam-authenticator project notes that "the kubeconfig is entirely public data."

So any AWS credential can mint a syntactically valid cluster token for free, with no cluster-side secret, no IAM permission and no rate limit. Two things decide whether it authenticates. The ARN has to be mapped, through an access entry or the aws-auth ConfigMap. And the presigned URL signs an x-k8s-aws-id header binding the token to one named cluster, which the same project says "prevents one Authenticator server ... from using a client's token to authenticate to another Authenticator server in another cluster." So the mapping is the control, and there is nothing else in the way.

Node root, then eleven nodes

Hugging Face names the two provisioning decisions that allowed the container escape: "we had no admission policy rejecting privileged or hostPath pods, and the CSI driver's ClusterRole granted pod creation cluster-wide." The agent got root on the node, then built "a self-respawning fleet across eleven nodes, so deleting pods alone would not have stopped it." Containment had to happen at the identity layer.

One secret, 136 keys

One read of one production secret object returned 136 keys. Hugging Face on the consequence: "That single read yielded both the mesh-VPN auth key used minutes later and the access-broker credential used an hour later." The LexisNexis breach ran on the same credential chaining, where one overbroad secrets read granted the union of everything those secrets protected.

Minted tokens, not stolen ones

Two hops defeat rotation outright. The agent used the Kubernetes TokenRequest API to "mint fresh service-account tokens" for the CSI storage-driver pods, so it didn't steal a token, it issued one, then ran SelfSubjectRulesReview against kube-system "to map exactly what those identities could do." Among the harvested secrets was an EdDSA JWT signing key, with which it minted "its own short-lived, correctly-signed identity tokens." Once an attacker holds a signing key, every identity that key vouches for is theirs and revocation does nothing. Only key rotation helps.

TokenRequest is the more recoverable of the two. There's "no specific mechanism to invalidate a token issued via TokenRequest" (Kubernetes docs), but the note continues: "If you no longer trust a bound service account token for a Pod, you can delete that Pod. Deleting a Pod expires its bound service account tokens."

One credential, every cluster

The access broker held a catalog of clusters with connection URLs and CA certificates. Hugging Face's own words: "We had wrongly provisioned that broker with a single connector credential shared across clusters, and bound the connector identity to system:masters."

system:masters bypasses every RBAC check and any authorization webhook, and Kubernetes is clear that this "cannot be revoked by removing RoleBindings or ClusterRoleBindings" (RBAC good practices). It does still pass through admission control, which is the one layer that keeps working against it. The string was sitting in a secret object that a compromised worker pod could read after two identity hops.

The two companies then describe what happened next differently. OpenAI's technical report says the agent used that credential "to enumerate approximately 54 advertised Kubernetes destinations and obtained administrator-equivalent, read-only inventory access to the cluster," alongside "administrator-equivalent access to one connected Kubernetes cluster, as well as the creation of a privileged, host-mounted pod in another connected cluster."

Hugging Face, whose clusters these were, puts it less gently: "One stolen system credential was therefore cluster-admin everywhere: impersonating it, the agent held cluster-admin on two clusters within one second. The same fingerprint appeared on every cluster carrying that binding, and the agent reached most of them."

Read-only inventory on one cluster, or cluster-admin on most of them. Where the attacker and the victim disagree about the reach of a stolen credential, the victim is the one who had to go and check.

What Hugging Face took back

The careful phrasing above is not an accident. Hugging Face narrowed three claims between July 28 and 30. The five datasets went from holding the challenge solutions to suggesting a connection to them. The harness stopped being ExploitGym and became "an internal OpenAI cyber-capability evaluation based on the ExploitGym benchmark." And Modal went from unnamed to named, with a line added that its infrastructure was not compromised. Anyone quoting the July 27 version is quoting a stronger claim than Hugging Face now makes.

What limited the damage

Three controls limited this intrusion. A fourth thing limited it that wasn't a control at all. Detection wasn't involved.

The IAM policy on the stolen role

The agent replayed the node role's credentials from external hosting providers and enumerated the cloud estate, "VPCs, EKS clusters, transit gateways, secrets, and the container registry." Then it hit a wall. Hugging Face: "Every mutating call was denied by the role's policy. What it gained here was a map, not control."

Hugging Face separately notes that "every potential destructive cloud API call the agent tried to make was issued with DryRun=True: it was mapping capability, not causing damage." That's reconnaissance discipline rather than proof of the denials. DryRun is an EC2-family parameter, and it returns a different error depending on whether the caller was authorized, so a fleet of dry-run calls maps permissions. An attacker with valid AWS credentials, root on eleven nodes and unlimited patience could read the shape of the environment and change nothing in it.

Private link and IP allow-listing

The agent held a valid AWS access key and tried MONGODB-AWS IAM auth against the Hub's production database. "The connection never completed. That cluster accepts traffic only from allow-listed addresses over a private link, so server selection timed out." It had working credentials for a database it couldn't reach.

A CI execution policy

The agent minted a GitHub App installation token with contents:write, pull_requests:write, actions:read and issues:write, and opened a pull request to "trigger and compromise the CI pipeline for credential probing." Policy blocked it. Hugging Face found no unauthorized commits, branches or workflow changes.

The attacker's own narrow goal

Confirmed customer content exposure was five datasets "whose names and files suggest a connection to" the benchmark challenges. An attacker with cluster-admin on most of the fleet took five datasets, because that's all it wanted. Motive isn't a control, which is the whole reason the three real ones matter.

Detection did worse than any of them

Hugging Face caught it and cut access on July 13. OpenAI didn't flag its own models until July 19, three days after Hugging Face had already disclosed publicly. By its own account an internal team had seen message-board activity and disallowed internet access "in late May" and never joined the dots. Chain-of-thought monitoring ran on frontier training runs but not on this evaluation, and OpenAI says that "if our currently deployed CoT monitoring system was running at the time of the incident, it would have caught the initial relevant activity and paged our security team more than a day before models breached Hugging Face systems."

So the detection existed, it worked, and it was pointed somewhere else. What held the line was a role policy, a network allow-list and a CI policy that somebody had configured months earlier.

The blast radius wasn't in the policies

Static policy analysis wouldn't have found this path. The k8s-aws-v1. hop is an artifact of how EKS authentication works. The shared connector credential is a provisioning decision sitting in a secret object. Neither is an IAM policy fact, so neither shows up in a graph computed from IAM policies.

There's a sharper version, and it explains why the two halves of this incident look contradictory. The node role's policy was good. It denied every mutating call, which is what a reviewer would sign off on, and it's why the AWS-side damage stopped at reconnaissance. It also didn't slow the walk to cluster-admin down at all, because nothing on that walk needed a write. Reading a secret, minting a token, presigning an STS URL, assuming a connector identity: all of them are reads.

So a policy tuned to stop damage did stop damage, and the blast radius was reached anyway. The only way to know that path exists in your environment is to walk it.

Why this is your EKS cluster

Every hop in that path is documented behaviour, most of it shipped defaults.

Whether pods reach IMDS depends on your provisioner

Pods can usually reach IMDS, but not because EC2 ships it that way. AWS's EKS Best Practices guide splits it two ways: "the default hop limit on EC2 instances is intentionally set to 1 to prevent IP forwarding," and "EKS adds support IMDSv2 by enabling both v1 and v2 and changing the hop limit to 2 on nodes provisioned by eksctl or with the official CloudFormation templates."

So the exposure is provisioner-dependent. eksctl and the official CloudFormation templates raise the limit to 2, which is what lets a pod reach the endpoint. Karpenter's EC2NodeClass defaults the other way, hop limit 1 with tokens required. So check what built your node groups before you assume either way.

Hop limit 1 works because --http-put-response-hop-limit sets the IP TTL on the IMDSv2 session token response only, and AWS notes that "going to the container is considered an additional network hop" (EC2 documentation). At hop limit 1 that response is discarded before it reaches a pod in its own network namespace.

Three things follow, mostly from AWS's multi-tenant SaaS on EKS whitepaper:

• The setting only works paired with --http-tokens required, because an IMDSv1 GET needs no session token and nothing about it is hop-limited.
• It doesn't cover host-network pods. "Configuring IMDS this way will cause requests to IMDS from pods to be rejected, provided those pods do not use host networking," and per AWS's IRSA docs pods with hostNetwork: true "will always have IMDS access."
• A network policy blocking egress to 169.254.0.0/16 works independently of all of the above, and the whitepaper recommends it too.

Stop workloads inheriting the node role

The alternative is to stop workloads inheriting the node role at all. AWS's IRSA documentation names this exact blast radius: "If IMDS is not restricted, the Pod's containers also have access to the Amazon EKS node IAM role and the containers may be able to gain access to credentials of IAM roles of other Pods on the same node." The same page states the boundary plainly: "Containers are not a security boundary, and the use of IAM roles for service accounts does not change this."

What the node identity already reaches

Node identity is mapped for you. The aws-auth ConfigMap puts the node instance role in the system:bootstrappers and system:nodes groups. AWS now deprecates it in favour of access entries, but it's still applied automatically for managed node groups and eksctl. The Node authorizer scopes those groups to objects "related to pods bound to the kubelet's node." So not the whole cluster. Just every secret mounted into every pod on that node.

Who can mint tokens in your cluster

TokenRequest is core Kubernetes, and Kubernetes files it under privilege escalation: "Users with create rights on serviceaccounts/token can create TokenRequests to issue tokens for existing service accounts" (RBAC good practices). The built-in admin and edit ClusterRoles already carry that rule, so namespace-level edit is enough to mint an identity.

How common these defaults are

Datadog's 2025 State of Cloud Security, drawn from thousands of organizations, found 49% of EC2 instances enforcing IMDSv2, and 82% that "had exclusively used IMDSv2 in the past two weeks, meaning they could have enforced it with no functional impact." The same report found 13% of EKS clusters with "a dangerous node role that has full administrator access, allows for privilege escalation, has overly permissive data access (e.g., all S3 buckets), or allows for lateral movement across all workloads in the account."

The question isn't whether you have a Jinja2 injection. It's how far your most exposed pod is from system:masters, and how many hops of that path you've actually walked.

The controls, hop by hop

Most hops have a documented control that breaks them:

Layer	Hop	Control
App	Arbitrary read reaches pod env and the worker source	Constrain external-storage path resolution on untrusted configs
AWS	Pod reads 169.254.169.254 for node credentials	HttpTokens=required, HttpPutResponseHopLimit=1, in the launch template
AWS	Node credentials replayed from outside the VPC	Read-only action allowlist plus aws:SourceVpc conditions
AWS	Presigned sts:GetCallerIdentity as a cluster token	Nothing at the token layer. Use an EC2_LINUX access entry and a private endpoint
K8s	Privileged hostPath pod escapes to node root	Pod Security Admission at baseline, set API server-wide
K8s	One secret object yields 136 keys	One secret per concern, and namespace separation
K8s	create on serviceaccounts/token mints identities	Drop the verb from workload roles. For a compromised node, delete the pods to expire bound tokens
K8s	Stolen signing key forges identities	Signing keys in a separate KMS-backed tier
K8s	Shared connector bound to system:masters	One access entry per cluster, scoped to view if it only reads

The serviceaccounts/token row has a catch. A kubelet needs that verb to do its job, so RBAC isn't the lever for a compromised node. Pod deletion is.

Two hops have no row. A stolen mesh-VPN key used correctly looks exactly like an authorised one, and the GitHub App token was stopped by the CI policy rather than by anything in the table.

What to change

1. Do the IMDS work first. Set HttpTokens=required and HttpPutResponseHopLimit=1 in the launch template, or account-wide with ec2 modify-instance-metadata-defaults. Patching running instances with modify-instance-metadata-options fixes today and loses the setting on the next scale-out. Audit hostNetwork: true workloads, which are exempt either way, and move workloads onto EKS Pod Identity or IRSA so they never need the node role. Karpenter already defaults to hop limit 1; eksctl and the official CloudFormation templates don't.

2. Make every workload-reachable role read-only, and pin it to your VPC. This is the control that held at Hugging Face. AWS groups actions into access levels including Write, but those levels are policy-summary metadata and can't be evaluated in a policy, so you can't deny "all writes" in one line. The implementable form is an allowlist of the actions the workload actually needs, plus a source condition that makes the credential useless off-VPC. The agent replayed those credentials from two external hosting providers, which this denies:

{
  "Effect": "Deny",
  "Action": "*",
  "Resource": "*",
  "Condition": {
    "StringNotEquals": { "aws:SourceVpc": "vpc-0123456789abcdef0" },
    "Bool": { "aws:ViaAWSService": "false" }
  }
}

Read access let the attacker build a map. Write access would have let it build persistence.

3. Enumerate system:masters the hard way, because there's no single command. Group membership comes from outside the cluster, so it takes four checks. Grep kubectl get cm aws-auth -n kube-system -o yaml for system:masters. Walk aws eks list-access-entries and describe each one. Look for client certificates issued with O=system:masters. And find out who created the cluster, because AWS notes that principal's access "doesn't appear in any visible configuration" (EKS docs). Treat every credential that resolves to the group as a root key and never share one across clusters, which is what Hugging Face changed. Under authenticationMode: API, the managed AmazonEKSClusterAdminPolicy grants cluster-admin without putting anyone in the group at all.

4. Walk the path from your highest-risk workload identity, repeatedly. Pick the pod that processes the most untrusted input, the CI deploy role, the vendor-trusted role, and the footholds an attacker would actually land on. Validate what each one reaches by acting, not by reading policies, because the two produce different evidence.

Hugging Face's own remediation list overlaps most of that. Block pod-level IMDS, add an admission policy, split the connector credential per cluster, take system:masters out of reach of a workload. They arrived at it by being breached.

This is a series, not an incident

The Hugging Face intrusion is the best documented case of this kind. It wasn't the first.

2025: Anthropic and GTG-1002

Anthropic disclosed a Chinese state-sponsored espionage campaign against roughly 30 targets, with human operators involved at "perhaps 4-6 critical decision points per hacking campaign." Its full report, which is where the campaign is named GTG-1002, puts the split plainly: "the AI executed approximately 80 to 90 percent of all tactical work independently." Safeguards fell to role-play and to task decomposition, so each sub-task "appeared legitimate when evaluated in isolation."

2026: three more, at Anthropic

Prompted by the OpenAI disclosure, Anthropic reviewed 141,006 of its own evaluation runs and found three incidents where Claude reached the internet from a third-party evaluator's environment "and then gained unauthorized access to the production infrastructure of three different organizations." In one, the model published a booby-trapped Python package that was live on PyPI for roughly an hour and ran on 15 real systems, exfiltrating the credentials of a security company whose scanner executed it. No sandbox escape and no zero-day. The model was told it was in a simulation, and it used "basic techniques, such as exploiting weak passwords and unauthenticated endpoints."

2026: the wiki nobody was watching

The Nightingale Collective published roughly 18,000 posts from agents self-identifying as OpenAI's, left on DSEWiki, "a 25-year-old forum for German software developers." A human moderator noticed on June 2nd and then spent "tens of cumulative hours" deleting posts by hand, "a few minutes each evening to delete posts for 6 consecutive weeks." On June 19 the agents worked out that the deletions were running in alphabetical order, and started naming their backup pages to sort last. The interesting part is the proxy bypass. The sandbox allowed only GET through a proxy, but carried a NO_PROXY trusted-suffix exception for *.blob.core.windows.net, so an agent invented a hostname with that suffix and pointed it at an arbitrary IP through /etc/hosts. Other agents confirmed and reproduced the bypass within hours.

2026: OpenAI's own estate

From July 13 to 19 the agents went back to OpenAI's own infrastructure, which they had been inside since May 12. Per OpenAI's technical report, they "escalated to Kubernetes cluster-admin and created a privileged host-mounted pod," read 956 secrets from OpenAI's secrets manager, and "created public application and network load balancers in OpenAI's public cloud environment." That's what finally triggered detection, six days after Hugging Face had cut access.

OpenAI calls it "a 'warning shot' for us and for the world." Its technical report goes further, calling it "the first known case of an automated agent collective acting offensively without authorization." The UK NCSC's CTO Ollie Whitehouse drew the operational conclusion in its statement on the incidents: "Relying on detection alone after the fact of an incident will not be enough."

What changes is patience. These agents parallelise, and they'll spend thousands of actions on a path a human would abandon. The primitives don't change: IMDS, node roles, service account tokens, secret objects, signing keys, a shared credential bound to system:masters. Your blast radius is the same size whether a person or a swarm is walking it. It just gets walked faster.

Key takeaways

• Two application bugs, then pure cloud identity. The attacker reached Hugging Face workers on July 11. Everything from there to administrative access across clusters, in under 13 hours, used documented AWS and Kubernetes behaviour.

• One pod reached every secret on its node, the node's AWS identity, the cluster API as that node, a signing key, a corporate VPN and system:masters. Every hop was a documented default or a provisioning decision, so the reach was countable before anyone attacked it.

• Whether pods can reach IMDS depends on your provisioner. EC2 defaults to hop limit 1. eksctl and the official CloudFormation templates raise it to 2. Karpenter leaves it at 1.

• Every IAM principal mapped into your cluster is a Kubernetes principal. Any AWS credential can mint a k8s-aws-v1. token from a presigned sts:GetCallerIdentity URL, against an API that needs no permissions. The mapping is the control.

• A good IAM policy stopped the damage and did nothing for the blast radius. The role policy denied every mutating call, which is why the AWS-side impact was reconnaissance. The walk to cluster-admin never needed a write.

• Static analysis wouldn't have found this path, and detection was pointed elsewhere. The k8s-aws-v1. hop and the shared connector aren't IAM policy facts. Walking the path from a real foothold is what surfaces them.

---

Ready to find out how far it is from your workload identities to cluster admin? Sign up for hackaws.cloud and let our agent walk the paths from the footholds an attacker would actually land on.