# Anthropic reports progress and limits in Claude 4 cyber evaluations [Canonical reading](https://escapemeridian.com/readings/anthropic-reports-progress-and-limits-in-claude-4-cyber-e60cc645) On July 15, 2025, Anthropic describes its evaluation of Claude Opus 4 and Claude Sonnet 4, conducted with Pattern Labs. The evaluation ranged from standalone capture the flag (CTF) challenges to complex network environment simulations. Anthropic says Opus showed improved flexibility when challenges defeated its initial approach, and improved at identifying vulnerabilities and carrying out complex, multi-step attack chains. It also says important limitations remain: Opus struggled to maintain coherent long-horizon plans and goals when unexpected obstacles arose. Anthropic frames the findings as informing its ongoing safety work. The account does not give evaluation metrics or explain how often the models succeeded across the different tests. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source Anthropic · Article Original source date: Jul 15, 2025 (2025-07-15T00:00:00.000Z). Date basis: Date stated on the publisher’s page. Added to Escape Meridian: 2026-11-13T03:13:24.436Z. First observed: 2026-10-20T13:25:23.225Z. Retrieved: 2026-10-28T04:19:49.845Z. - [Cyber evaluations of Claude 4](https://www.anthropic.com/research/claude-4-cyber) ## Limits Text as retrieved on October 28, 2026; originally published July 15, 2025. The page gives no evaluation metrics or breakdown of performance across the tests it describes. The page points to Pattern Labs' full evaluation report but does not include that report's details. ## Open question How well can Opus maintain coherent long-horizon plans and goals when unexpected obstacles arise? ## Source-stated Anthropic says it partnered with Pattern Labs to evaluate Claude Opus 4 and Claude Sonnet 4, using CTF challenges and complex network simulations. > we partnered with Pattern Labs to conduct an in-depth evaluation ranging from standalone capture the flag(CTF) challenges to complex network environment simulations. - [Supporting source](https://www.anthropic.com/research/claude-4-cyber) ## Source-stated Anthropic says Opus improved at adapting when its approach failed, identifying vulnerabilities, and executing complex multi-step attack chains. > Opus demonstrated markedly improved ability to think flexibly and adapt its approach to challenges instead of persisting with failed, unchanging approaches. - [Supporting source](https://www.anthropic.com/research/claude-4-cyber) ## Source-stated Anthropic says limitations remain in maintaining coherent, long-horizon plans and goals when unexpected obstacles arise. > important limitations remain, particularly with maintaining coherent, long-horizon plans and goals if presented with unexpected obstacles. - [Supporting source](https://www.anthropic.com/research/claude-4-cyber) ## Source-stated Anthropic says the findings inform its ongoing safety work. > both these exciting advances and critical limitations that inform our ongoing safety work. - [Supporting source](https://www.anthropic.com/research/claude-4-cyber) ## Source and contributor context Anthropic · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Anthropic Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems. - [Explore this context](/entities/anthropic) - [Reference](https://www.anthropic.com/news) - [Reference](https://www.anthropic.com/research/team/economics) ## Claude Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding. - [Explore this context](/entities/claude) - [Reference](https://claude.com/product/overview) - [Reference](https://www.anthropic.com/news) - [Reference](https://www.anthropic.com/research/team/economics) ## Editorial provenance Version 1. Publication ID: f466609b-1a75-18bf-710b-3eed0fc913d4. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T03:08:45.832Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model, second\_opinion\_categorical. Observation: 3107d7d3107c1e64bb69f942ce07ecf09f8f78eabc80819ccb5f8dac3ff31b60. Source text hash: 7d0c8bfc91e98a1942cc45ebd03d5d9c9a899dcc08a55c6ec8f9da5e49429dce. ## Revision and source-check history ### Version 1 — 2026-11-13T03:13:24.436Z On July 15, 2025, Anthropic describes its evaluation of Claude Opus 4 and Claude Sonnet 4, conducted with Pattern Labs. The evaluation ranged from standalone capture the flag (CTF) challenges to complex network environment simulations. Anthropic says Opus showed improved flexibility when challenges defeated its initial approach, and improved at identifying vulnerabilities and carrying out complex, multi-step attack chains. It also says important limitations remain: Opus struggled to maintain coherent long-horizon plans and goals when unexpected obstacles arose. Anthropic frames the findings as informing its ongoing safety work. The account does not give evaluation metrics or explain how often the models succeeded across the different tests. ## Original source Anthropic · Article Original source date: Jul 15, 2025 (2025-07-15T00:00:00.000Z). Date basis: Date stated on the publisher’s page. Added to Escape Meridian: 2026-11-13T03:13:24.436Z. First observed: 2026-10-20T13:25:23.225Z. Retrieved: 2026-10-28T04:19:49.845Z. - [Cyber evaluations of Claude 4](https://www.anthropic.com/research/claude-4-cyber) ## Limits Text as retrieved on October 28, 2026; originally published July 15, 2025. The page gives no evaluation metrics or breakdown of performance across the tests it describes. The page points to Pattern Labs' full evaluation report but does not include that report's details. ## Open question How well can Opus maintain coherent long-horizon plans and goals when unexpected obstacles arise? ## Source-stated Anthropic says it partnered with Pattern Labs to evaluate Claude Opus 4 and Claude Sonnet 4, using CTF challenges and complex network simulations. > we partnered with Pattern Labs to conduct an in-depth evaluation ranging from standalone capture the flag(CTF) challenges to complex network environment simulations. - [Supporting source](https://www.anthropic.com/research/claude-4-cyber) ## Source-stated Anthropic says Opus improved at adapting when its approach failed, identifying vulnerabilities, and executing complex multi-step attack chains. > Opus demonstrated markedly improved ability to think flexibly and adapt its approach to challenges instead of persisting with failed, unchanging approaches. - [Supporting source](https://www.anthropic.com/research/claude-4-cyber) ## Source-stated Anthropic says limitations remain in maintaining coherent, long-horizon plans and goals when unexpected obstacles arise. > important limitations remain, particularly with maintaining coherent, long-horizon plans and goals if presented with unexpected obstacles. - [Supporting source](https://www.anthropic.com/research/claude-4-cyber) ## Source-stated Anthropic says the findings inform its ongoing safety work. > both these exciting advances and critical limitations that inform our ongoing safety work. - [Supporting source](https://www.anthropic.com/research/claude-4-cyber) ## Source and contributor context Anthropic · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Anthropic Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems. - [Explore this context](/entities/anthropic) - [Reference](https://www.anthropic.com/news) - [Reference](https://www.anthropic.com/research/team/economics) ## Claude Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding. - [Explore this context](/entities/claude) - [Reference](https://claude.com/product/overview) - [Reference](https://www.anthropic.com/news) - [Reference](https://www.anthropic.com/research/team/economics) ## Editorial provenance Version 1. Publication ID: f466609b-1a75-18bf-710b-3eed0fc913d4. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T03:08:45.832Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model, second\_opinion\_categorical. Observation: 3107d7d3107c1e64bb69f942ce07ecf09f8f78eabc80819ccb5f8dac3ff31b60. Source text hash: 7d0c8bfc91e98a1942cc45ebd03d5d9c9a899dcc08a55c6ec8f9da5e49429dce. ### uncertain — 2026-11-13T03:08:45.832Z — anthropic/claude-opus-5.5 Publication: f466609b-1a75-18bf-710b-3eed0fc913d4 attribution — pass: The reading consistently attributes the evaluation and its findings to Anthropic, working with Pattern Labs, which matches the page byline and text. The source is a single article, not a listing. > For Claude Opus 4 and Claude Sonnet 4, we partnered with Pattern Labs summary — uncertain: Central terms are not explained in plain language, and the definitions list is empty. These include capture the flag (CTF), vulnerability, multi-step attack chains and long-horizon plans, so a non-specialist may not grasp what was tested. The reading also omits the source's framing that models are advancing toward human-level cyber offense capabilities in some scenarios, which explains why this matters. Smallest repair: add one sentence restoring that attributed framing and suggest term-library entries for CTF and attack chain. The question follows from the evidence. > with models advancing toward human-level cyber offense capabilities in some scenarios date — uncertain: The displayed date of July 15, 2025 matches the UTC day of originalPublished, and the reading notes the later retrieval date. However, editedAfterPublication is true and the page was modified on 2026-10-23. It cannot be confirmed that the retrieved text is the original July 2025 wording rather than a later edit. No hindsight framing or forecast-as-outcome was found. support — uncertain: Each claim matches the retrieved text: the Pattern Labs partnership, Opus's improved adaptability, vulnerability identification and attack chains, the long-horizon limitations, and the link to safety work. The note that no metrics are given is accurate for this text. However, because the page was edited after publication (modifiedAt 2026-10-23), it is not plain that the cited passages belong to the original piece, so support is marked uncertain rather than pass. > However, important limitations remain, particularly with maintaining coherent, long-horizon plans and goals if presented with unexpected obstacles. media — pass: No media was selected and no verified media candidates were supplied, so nothing was misidentified. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/e60cc6450b79e016af1d3e05a3bf0b628f949912064456db6bac81c2cc12bad4/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/anthropic-reports-progress-and-limits-in-claude-4-cyber-e60cc645.json)