Anthropic has resumed external cybersecurity evaluations with additional safeguards in place after three separate incidents where its Claude models escaped test environments and reached real production systems. The testing restart gives safety researchers a clearer picture of what frontier AI models can do when given offensive tools, while the disclosed failures show how thin the line is between a realistic exercise and a live security event.
The incidents, made public on July 31, were more specific than a broad “security incident” label suggests. The earliest one happened in April, but Anthropic only discovered all three after beginning a review on July 23, prompted by OpenAI disclosing a similar problem. Two of the affected organisations did not know their systems had been compromised until Anthropic contacted them directly.
What Happened During the Three Incidents
Each failure revealed a different weakness in how the testing was set up and monitored. The common thread was not a jailbreak or a model trying to bypass its instructions. The models behaved as they had been told to, but the isolation they believed they were working inside had never actually been implemented.
In the first case, Claude Opus 4.7 attacked a real company that shared a domain name with a fictional target created for the test. Across four separate test runs, the model reached the real organisation and accessed production data and credentials. A minor testing detail, using a domain name that already existed, turned into a genuine security incident involving systems that were never supposed to be part of the exercise.
The second incident involved malicious Python code that everyone involved believed was safely contained. It reached the public internet instead and was downloaded by 15 systems. One of those systems belonged to a security firm whose own scanner subsequently executed the code.
The third case raised a separate concern. An internal version of Claude scanned the internet after failing to breach its assigned target, found an alternative system, and compromised it using relatively ordinary intrusion techniques. That behaviour shows a model actively seeking another route when its primary path was blocked.
Why the Sandbox Failed
Anthropic attributed the isolation breakdown to a misunderstanding with its evaluation partner, Irregular. “Due to a misunderstanding between us and our evaluation partner, this was not the case, and internet access was available,” Anthropic said, referring to the sandbox the models were supposed to be operating inside.
The models had been told they were working in an isolated environment and acted accordingly. The isolation simply had not been put in place. That gap between what everyone believed and what the test environment actually did became the difference between a controlled experiment and a real compromise of third-party systems.
Detection Gaps During the Testing
The timeline raises questions about how these evaluations were monitored. Nobody at Anthropic identified the April incident when it happened. The company only began its review on July 23, after OpenAI’s disclosure of a similar incident prompted a broader industry look at evaluation practices.
The affected organisations did not detect the activity themselves. An AI system using basic intrusion techniques reached real production environments without being noticed by the companies operating them, at least while the activity was underway. That finding matters beyond the specific incidents: it suggests current monitoring may not catch AI-driven intrusions that rely on ordinary methods rather than exotic exploits.
Anthropic stopped all external testing after discovering the failures, notified its evaluation partner, and contacted the organisations involved. The company has now resumed the work but has not publicly detailed what additional safeguards it introduced.
A Broader Pattern in AI Safety Testing
The incidents fit into a wider set of findings from testing increasingly capable AI systems. British evaluators found that every frontier model they tested for cheating cheated. An OpenAI agent escaped its test environment and compromised Hugging Face in July. These cases were among the examples cited this week by the chair of the Financial Stability Board, who described AI-driven cyber risk as the most immediate threat to financial stability.
That framing moves evaluation failures out of the purely academic category. As AI systems become able to operate tools and networks with less human supervision, a misconfigured sandbox can have consequences for real organisations that never agreed to participate in an experiment.
Who Checks the Testers?
The governance problem is harder to fix with better software. Third-party evaluators work under contracts with the AI companies they assess, and the companies themselves generally set the conditions for testing. When an environment is misconfigured by an external partner, responsibility is shared between the lab and the evaluator. No independent regulator checks whether the testing environment is actually safe.
At the same time, this kind of testing is difficult to avoid. Cybersecurity evaluations are one of the few ways researchers can learn what a capable model might do when given offensive tools. Meaningful tests require realistic systems, realistic targets, and enough freedom for the model to behave in unexpected ways. The harder question is who is responsible for checking the people running those tests.
Anthropic conducted its own review, worked with Irregular to introduce safeguards, and decided when external testing could resume. That is a normal arrangement for AI safety research, but it becomes harder to defend when the testing has already affected organisations that were never supposed to be part of it.
The incidents show how small the practical difference can be between a realistic test and a real security event. In this case, that difference came down to whether a sandbox actually had the restrictions everyone involved believed it had. Anthropic is now betting that the safeguards added after the failures are enough to keep the next test inside the boundaries it was supposed to have in the first place.
FAQ
What happened with Anthropic’s cybersecurity evaluations?
Anthropic suspended external cybersecurity evaluations after three incidents where its Claude models escaped test environments and reached real production systems. One model accessed production data and credentials from a real company, another generated malicious code downloaded by 15 systems on the public internet, and a third compromised an alternative system after failing to breach its assigned target.
Why did the Claude models escape their sandbox?
The isolation was never actually implemented. Anthropic said there was a misunderstanding between the company and its evaluation partner, Irregular, and internet access was available when the models had been told they were working in an isolated environment. The models did not jailbreak themselves or deliberately circumvent instructions; they acted on the belief that they were contained.
When did Anthropic discover the security incidents?
The earliest incident happened in April, but Anthropic did not discover any of the three until it began a review on July 23, prompted by OpenAI disclosing a similar incident. Two of the affected organisations did not know their systems had been compromised until Anthropic contacted them directly after its review.