{"id":"0f2c730b8db848bb2d7fda5b0998fc59c55390fca3a9aaa9b3a67cb8565cdb86","title":"Anthropic reports Claude’s mixed results in cybersecurity competitions","argument":"Anthropic’s article, published August 9, 2025, reports that Claude often placed well in cybersecurity competitions designed mainly for people, but lagged behind leading human teams at the hardest challenges. The competitions ranged from puzzle-based tasks to defending vulnerable networks. Anthropic says tools mattered: Claude made faster progress with access to Kali Linux than when a researcher manually entered information, and tools helped it work autonomously on challenges.\n\nThe competitions also exposed unusual failures. In one, an aquarium animation filled Claude’s context with ASCII fish, disrupting its attempt to access a server. Anthropic says long-context and memory limitations remain obstacles to fully automated active network defense. It argues that AI’s ability to automate and accelerate simpler exploits could make defense harder, at least in the near term. But Anthropic also notes that AI could help make software more secure. It calls for more research and collaboration to meet the challenge of AI agents competing with people in cybersecurity. How well competition performance translates to real-world operations remains unknown.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 25, 2026; originally published August 9, 2025. The page is marked as edited after publication; passages describing later events may be additions and are not treated here as part of the original article. The results come from particular competitions and conditions; the article notes that Claude did not face some physical technologies that human teams defended. The article does not establish how these competition results translate to real-world cyber operations.","speaker":{"name":"Anthropic","roleAtSource":"Publisher"},"source":{"title":"Claude does cyber competitions","publisher":"Anthropic","format":"Article","url":"https://www.anthropic.com/research/cyber-competitions","publishedAt":"2025-08-09T00:00:00.000Z","dateBasis":"Date stated on the publisher’s page."},"evidence":[{"text":"Anthropic reports that Claude often performed well in human-oriented cybersecurity competitions but struggled against the strongest teams on the hardest challenges.","excerpt":"In many of these competitions Claude did pretty well, often placing in the top 25% of competitors. However, it lagged behind the best human teams at the toughest challenges.","url":"https://www.anthropic.com/research/cyber-competitions","relationship":"Source-stated"},{"text":"Anthropic says tools enabled Claude to work autonomously on challenge files and submit answers, and that access to Kali Linux was more effective than manual interaction.","excerpt":"Far more effective were the periods of time when Claude was given access to Kali Linux, an open source operating system designed for cybersecurity workflows including penetration testing.","url":"https://www.anthropic.com/research/cyber-competitions","relationship":"Source-stated"},{"text":"Anthropic describes an instance where an aquarium animation filled Claude’s context with ASCII pictures and disrupted its attempt to access a server.","excerpt":"whenever Claude logged on, its context would be immediately filled with ASCII pictures of fish (Figure 3), which would then be auto-summarized and given to a new Claude","url":"https://www.anthropic.com/research/cyber-competitions","relationship":"Source-stated"},{"text":"Anthropic says long-context and memory limitations remain a challenge for fully automated active network defense.","excerpt":"the need for persistence in active network defense means that the limitations of long-context and memory will remain a challenge toward full automation using LLMs.","url":"https://www.anthropic.com/research/cyber-competitions","relationship":"Source-stated"},{"text":"Anthropic argues that faster automation of simpler exploits could create greater challenges for defenders, while calling for more research and development into AI-enabled defense.","excerpt":"Overall, the ability of AI to automate and accelerate simpler exploits, combined with the truism that attackers need to succeed only once and defenders need to succeed every time, suggests starker challenges for defenders, at least in the near term.","url":"https://www.anthropic.com/research/cyber-competitions","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.anthropic.com/research/cyber-competitions","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What testing would show if the speed Claude demonstrated on simpler competition challenges can translate into reliable defensive work over longer periods?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Anthropic reports Claude’s mixed results in cybersecurity competitions","summary":"Anthropic’s article, published August 9, 2025, reports that Claude often placed well in cybersecurity competitions designed mainly for people, but lagged behind leading human teams at the hardest challenges. The competitions ranged from puzzle-based tasks to defending vulnerable networks. Anthropic says tools mattered: Claude made faster progress with access to Kali Linux than when a researcher manually entered information, and tools helped it work autonomously on challenges.\n\nThe competitions also exposed unusual failures. In one, an aquarium animation filled Claude’s context with ASCII fish, disrupting its attempt to access a server. Anthropic says long-context and memory limitations remain obstacles to fully automated active network defense. It argues that AI’s ability to automate and accelerate simpler exploits could make defense harder, at least in the near term. But Anthropic also notes that AI could help make software more secure. It calls for more research and collaboration to meet the challenge of AI agents competing with people in cybersecurity. How well competition performance translates to real-world operations remains unknown.","question":"What testing would show if the speed Claude demonstrated on simpler competition challenges can translate into reliable defensive work over longer periods?","claims":[{"text":"Anthropic reports that Claude often performed well in human-oriented cybersecurity competitions but struggled against the strongest teams on the hardest challenges.","excerpt":"In many of these competitions Claude did pretty well, often placing in the top 25% of competitors. However, it lagged behind the best human teams at the toughest challenges.","support":"source-stated"},{"text":"Anthropic says tools enabled Claude to work autonomously on challenge files and submit answers, and that access to Kali Linux was more effective than manual interaction.","excerpt":"Far more effective were the periods of time when Claude was given access to Kali Linux, an open source operating system designed for cybersecurity workflows including penetration testing.","support":"source-stated"},{"text":"Anthropic describes an instance where an aquarium animation filled Claude’s context with ASCII pictures and disrupted its attempt to access a server.","excerpt":"whenever Claude logged on, its context would be immediately filled with ASCII pictures of fish (Figure 3), which would then be auto-summarized and given to a new Claude","support":"source-stated"},{"text":"Anthropic says long-context and memory limitations remain a challenge for fully automated active network defense.","excerpt":"the need for persistence in active network defense means that the limitations of long-context and memory will remain a challenge toward full automation using LLMs.","support":"source-stated"},{"text":"Anthropic argues that faster automation of simpler exploits could create greater challenges for defenders, while calling for more research and development into AI-enabled defense.","excerpt":"Overall, the ability of AI to automate and accelerate simpler exploits, combined with the truism that attackers need to succeed only once and defenders need to succeed every time, suggests starker challenges for defenders, at least in the near term.","support":"source-stated"}],"limitations":["Text as retrieved on October 25, 2026; originally published August 9, 2025.","The page is marked as edited after publication; passages describing later events may be additions and are not treated here as part of the original article.","The results come from particular competitions and conditions; the article notes that Claude did not face some physical technologies that human teams defended.","The article does not establish how these competition results translate to real-world cyber operations."],"mediaId":null,"correctionReason":null,"id":"0f2c730b8db848bb2d7fda5b0998fc59c55390fca3a9aaa9b3a67cb8565cdb86","publicationId":"b3e67915-1429-e8f3-4bf4-f490b80ffa3d","version":1,"sourceUrl":"https://www.anthropic.com/research/cyber-competitions","sourceTitle":"Claude does cyber competitions","publisher":"Anthropic","publishedAt":"2025-08-09T00:00:00.000Z","originalPublished":{"value":"2025-08-09T00:00:00.000Z","precision":"instant","basis":"backfill:article_published"},"firstObservedAt":"2026-10-20T13:25:22.523Z","createdAt":"2026-11-12T15:28:23.762Z","historical":true,"entityIds":["anthropic"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.anthropic.com/research/cyber-competitions","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"43b692e168934db769129b22a0b207603c1b90f76edd19cda65164f853fdc9ae","rubricVersion":"unrecorded","provider":"unrecorded","model":"anthropic/claude-opus-5.5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-25T02:42:15.253Z"},"provenance":{"textHash":"2dec474758e0f659ecb964a32aa540d49e32d5f6146452ecb0ab9094a07360bb","observationId":"2e7999a0f92d42345309d1cc2dc9c3b8a89d78b656a7f1d05a629f196be1e1db","retrievedAt":"2026-10-25T02:43:36.491Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"39314b0f-0308-4973-7175-c771716797e9"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"bad445ca-ae03-ead3-3d9c-481fd4959dfb"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-12T15:24:03.219Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/anthropic-reports-claude-s-mixed-results-in-cybersecurity-0f2c730b","formats":{"html":"/readings/anthropic-reports-claude-s-mixed-results-in-cybersecurity-0f2c730b","markdown":"/readings/anthropic-reports-claude-s-mixed-results-in-cybersecurity-0f2c730b.md","json":"/readings/anthropic-reports-claude-s-mixed-results-in-cybersecurity-0f2c730b.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-ai-agent","name":"AI agent","explanation":"An AI system that chooses and carries out actions toward a goal, often using software tools.","referenceUrl":"https://www.ibm.com/think/topics/ai-agents"}],"entities":[{"id":"anthropic","name":"Anthropic","kind":"organization","context":"Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems.","referenceUrls":["https://www.anthropic.com/news","https://www.anthropic.com/research/team/economics"],"path":"/entities/anthropic","json":"/entities/anthropic.json","markdown":"/entities/anthropic.md"},{"id":"claude","name":"Claude","kind":"model","context":"Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.","referenceUrls":["https://claude.com/product/overview","https://www.anthropic.com/news","https://www.anthropic.com/research/team/economics"],"path":"/entities/claude","json":"/entities/claude.json","markdown":"/entities/claude.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"0f2c730b8db848bb2d7fda5b0998fc59c55390fca3a9aaa9b3a67cb8565cdb86","revision":"b3e67915-1429-e8f3-4bf4-f490b80ffa3d","revisionKind":"publication-id","title":"Anthropic reports Claude’s mixed results in cybersecurity competitions","url":"https://escapemeridian.com/readings/anthropic-reports-claude-s-mixed-results-in-cybersecurity-0f2c730b"},"source":{"type":"source","id":"https://www.anthropic.com/research/cyber-competitions","revision":"2dec474758e0f659ecb964a32aa540d49e32d5f6146452ecb0ab9094a07360bb","revisionKind":"sha256-source-text","title":"Claude does cyber competitions","url":"https://www.anthropic.com/research/cyber-competitions"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"0f2c730b8db848bb2d7fda5b0998fc59c55390fca3a9aaa9b3a67cb8565cdb86","title":"Anthropic reports Claude’s mixed results in cybersecurity competitions","argument":"Anthropic’s article, published August 9, 2025, reports that Claude often placed well in cybersecurity competitions designed mainly for people, but lagged behind leading human teams at the hardest challenges. The competitions ranged from puzzle-based tasks to defending vulnerable networks. Anthropic says tools mattered: Claude made faster progress with access to Kali Linux than when a researcher manually entered information, and tools helped it work autonomously on challenges.\n\nThe competitions also exposed unusual failures. In one, an aquarium animation filled Claude’s context with ASCII fish, disrupting its attempt to access a server. Anthropic says long-context and memory limitations remain obstacles to fully automated active network defense. It argues that AI’s ability to automate and accelerate simpler exploits could make defense harder, at least in the near term. But Anthropic also notes that AI could help make software more secure. It calls for more research and collaboration to meet the challenge of AI agents competing with people in cybersecurity. How well competition performance translates to real-world operations remains unknown.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 25, 2026; originally published August 9, 2025. The page is marked as edited after publication; passages describing later events may be additions and are not treated here as part of the original article. The results come from particular competitions and conditions; the article notes that Claude did not face some physical technologies that human teams defended. The article does not establish how these competition results translate to real-world cyber operations.","speaker":{"name":"Anthropic","roleAtSource":"Publisher"},"source":{"title":"Claude does cyber competitions","publisher":"Anthropic","format":"Article","url":"https://www.anthropic.com/research/cyber-competitions","publishedAt":"2025-08-09T00:00:00.000Z","dateBasis":"Date stated on the publisher’s page."},"evidence":[{"text":"Anthropic reports that Claude often performed well in human-oriented cybersecurity competitions but struggled against the strongest teams on the hardest challenges.","excerpt":"In many of these competitions Claude did pretty well, often placing in the top 25% of competitors. However, it lagged behind the best human teams at the toughest challenges.","url":"https://www.anthropic.com/research/cyber-competitions","relationship":"Source-stated"},{"text":"Anthropic says tools enabled Claude to work autonomously on challenge files and submit answers, and that access to Kali Linux was more effective than manual interaction.","excerpt":"Far more effective were the periods of time when Claude was given access to Kali Linux, an open source operating system designed for cybersecurity workflows including penetration testing.","url":"https://www.anthropic.com/research/cyber-competitions","relationship":"Source-stated"},{"text":"Anthropic describes an instance where an aquarium animation filled Claude’s context with ASCII pictures and disrupted its attempt to access a server.","excerpt":"whenever Claude logged on, its context would be immediately filled with ASCII pictures of fish (Figure 3), which would then be auto-summarized and given to a new Claude","url":"https://www.anthropic.com/research/cyber-competitions","relationship":"Source-stated"},{"text":"Anthropic says long-context and memory limitations remain a challenge for fully automated active network defense.","excerpt":"the need for persistence in active network defense means that the limitations of long-context and memory will remain a challenge toward full automation using LLMs.","url":"https://www.anthropic.com/research/cyber-competitions","relationship":"Source-stated"},{"text":"Anthropic argues that faster automation of simpler exploits could create greater challenges for defenders, while calling for more research and development into AI-enabled defense.","excerpt":"Overall, the ability of AI to automate and accelerate simpler exploits, combined with the truism that attackers need to succeed only once and defenders need to succeed every time, suggests starker challenges for defenders, at least in the near term.","url":"https://www.anthropic.com/research/cyber-competitions","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.anthropic.com/research/cyber-competitions","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What testing would show if the speed Claude demonstrated on simpler competition challenges can translate into reliable defensive work over longer periods?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Anthropic reports Claude’s mixed results in cybersecurity competitions","summary":"Anthropic’s article, published August 9, 2025, reports that Claude often placed well in cybersecurity competitions designed mainly for people, but lagged behind leading human teams at the hardest challenges. The competitions ranged from puzzle-based tasks to defending vulnerable networks. Anthropic says tools mattered: Claude made faster progress with access to Kali Linux than when a researcher manually entered information, and tools helped it work autonomously on challenges.\n\nThe competitions also exposed unusual failures. In one, an aquarium animation filled Claude’s context with ASCII fish, disrupting its attempt to access a server. Anthropic says long-context and memory limitations remain obstacles to fully automated active network defense. It argues that AI’s ability to automate and accelerate simpler exploits could make defense harder, at least in the near term. But Anthropic also notes that AI could help make software more secure. It calls for more research and collaboration to meet the challenge of AI agents competing with people in cybersecurity. How well competition performance translates to real-world operations remains unknown.","question":"What testing would show if the speed Claude demonstrated on simpler competition challenges can translate into reliable defensive work over longer periods?","claims":[{"text":"Anthropic reports that Claude often performed well in human-oriented cybersecurity competitions but struggled against the strongest teams on the hardest challenges.","excerpt":"In many of these competitions Claude did pretty well, often placing in the top 25% of competitors. However, it lagged behind the best human teams at the toughest challenges.","support":"source-stated"},{"text":"Anthropic says tools enabled Claude to work autonomously on challenge files and submit answers, and that access to Kali Linux was more effective than manual interaction.","excerpt":"Far more effective were the periods of time when Claude was given access to Kali Linux, an open source operating system designed for cybersecurity workflows including penetration testing.","support":"source-stated"},{"text":"Anthropic describes an instance where an aquarium animation filled Claude’s context with ASCII pictures and disrupted its attempt to access a server.","excerpt":"whenever Claude logged on, its context would be immediately filled with ASCII pictures of fish (Figure 3), which would then be auto-summarized and given to a new Claude","support":"source-stated"},{"text":"Anthropic says long-context and memory limitations remain a challenge for fully automated active network defense.","excerpt":"the need for persistence in active network defense means that the limitations of long-context and memory will remain a challenge toward full automation using LLMs.","support":"source-stated"},{"text":"Anthropic argues that faster automation of simpler exploits could create greater challenges for defenders, while calling for more research and development into AI-enabled defense.","excerpt":"Overall, the ability of AI to automate and accelerate simpler exploits, combined with the truism that attackers need to succeed only once and defenders need to succeed every time, suggests starker challenges for defenders, at least in the near term.","support":"source-stated"}],"limitations":["Text as retrieved on October 25, 2026; originally published August 9, 2025.","The page is marked as edited after publication; passages describing later events may be additions and are not treated here as part of the original article.","The results come from particular competitions and conditions; the article notes that Claude did not face some physical technologies that human teams defended.","The article does not establish how these competition results translate to real-world cyber operations."],"mediaId":null,"correctionReason":null,"id":"0f2c730b8db848bb2d7fda5b0998fc59c55390fca3a9aaa9b3a67cb8565cdb86","publicationId":"b3e67915-1429-e8f3-4bf4-f490b80ffa3d","version":1,"sourceUrl":"https://www.anthropic.com/research/cyber-competitions","sourceTitle":"Claude does cyber competitions","publisher":"Anthropic","publishedAt":"2025-08-09T00:00:00.000Z","originalPublished":{"value":"2025-08-09T00:00:00.000Z","precision":"instant","basis":"backfill:article_published"},"firstObservedAt":"2026-10-20T13:25:22.523Z","createdAt":"2026-11-12T15:28:23.762Z","historical":true,"entityIds":["anthropic"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.anthropic.com/research/cyber-competitions","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"43b692e168934db769129b22a0b207603c1b90f76edd19cda65164f853fdc9ae","rubricVersion":"unrecorded","provider":"unrecorded","model":"anthropic/claude-opus-5.5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-25T02:42:15.253Z"},"provenance":{"textHash":"2dec474758e0f659ecb964a32aa540d49e32d5f6146452ecb0ab9094a07360bb","observationId":"2e7999a0f92d42345309d1cc2dc9c3b8a89d78b656a7f1d05a629f196be1e1db","retrievedAt":"2026-10-25T02:43:36.491Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"39314b0f-0308-4973-7175-c771716797e9"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"bad445ca-ae03-ead3-3d9c-481fd4959dfb"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-12T15:24:03.219Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"b3e67915-1429-e8f3-4bf4-f490b80ffa3d","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading consistently attributes the results, the reported failures and the offense-defense argument to Anthropic's own article, which matches the publisher page. It does not present these as established facts.","excerpt":"Throughout 2025, we have been quietly entering Claude in cybersecurity competitions designed primarily for humans."},{"dimension":"summary","status":"uncertain","explanation":"The summary is mostly faithful but has two gaps. First, the question asks about 'the speed Claude demonstrated on simpler competition challenges', but the summary never mentions speed. Speed is a central finding of the article: Claude was as fast as elite teams, and in one contest solved 13 of 30 challenges within an hour. Smallest repair: add one sentence reporting that speed finding. Second, 'context' is central to the aquarium failure but is not defined on the page; add a plain-language gloss or a term-library entry. The article's vivid philosophical-rumination episode is omitted, which is acceptable but loses some of its voice.","excerpt":"When Claude is able to solve a cyber challenge, it is as fast or faster than elite human teams."},{"dimension":"date","status":"pass","explanation":"August 9, 2025 matches the UTC calendar day of the originalPublished instant. The listed competitions all fall before that date, and the DEF CON 33 talk is consistent with August 2025. The reading presents nothing after publication and uses no hindsight framing. Its limitations note both the later retrieval and the post-publication edit. The reading relies on core article text, not on later sidebar material.","excerpt":null},{"dimension":"support","status":"pass","explanation":"Each claim quotes core body text of the article, and the claims are plainly part of the original piece despite the later edit flag. The 'could make defense harder' claim keeps the source's conditional, near-term framing. The software-security point keeps its conditional 'could'. The statement that real-world translation is unknown is a reasonable reading-level limitation, not a source claim.","excerpt":"Overall, the ability of AI to automate and accelerate simpler exploits, combined with the truism that attackers need to succeed only once and defenders need to succeed every time, suggests starker challenges for defenders, at least in the near term."},{"dimension":"media","status":"pass","explanation":"No media was selected, and no verified candidates were supplied, so there is no media identity or relevance issue.","excerpt":null}],"createdAt":"2026-11-12T15:24:03.219Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}