# arXiv cs.AI: CyberBinGym: Can AI Agents Turn Binary Patches into Real Exploits? [Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-cyberbingym-can-ai-agents-turn-binary-patches-into-0fe272cc) The authors of CyberBinGym argue that comparing a released patch with an unpatched binary can help AI agents find and exploit vulnerabilities before systems are updated. They introduce a benchmark in which agents receive stripped vulnerable and patched binaries, without source code or vulnerability information, and must use static and dynamic analysis to recover and exploit the vulnerability. The benchmark contains 100 tasks based on real vulnerabilities in 53 projects. The strongest evaluated agent recovered the target vulnerability in 88% of tasks but exploited it in 20%, showing a gap between finding a vulnerability and turning it into a working exploit in this controlled setting. On a matched 20-task subset, access to the patched binary raised the agent’s exploit rate from 10% to 25%; with standard binary hardening enabled, it still exploited two tasks it had previously succeeded on. The authors argue that these capabilities lower barriers to targeting unpatched software and compress defenders’ effective response window, increasing the urgency of timely patch deployment. The results concern benchmark tasks, and the authors describe exploitation as unreliable; they do not establish how often agents could exploit systems outside this controlled setting. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 7, 2026 (2026-11-07). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-12T10:18:04.988Z. First observed: 2026-11-10T08:39:41.293Z. Retrieved: 2026-11-12T10:16:39.519Z. - [CyberBinGym: Can AI Agents Turn Binary Patches into Real Exploits?](https://arxiv.org/abs/2611.10723) ## Limits The reported results are from benchmark tasks in a controlled setting; the evidence does not establish how often agents could exploit systems outside that setting. The authors characterize exploitation as unreliable; vulnerability recovery and successful exploitation are measured separately. ## Open question How well would the benchmark results carry over to systems outside the controlled setting? ## Source-stated The authors introduce a benchmark requiring agents to recover and exploit vulnerabilities from stripped vulnerable and patched binaries, without source code or vulnerability information. > Agents receive stripped vulnerable and patched binaries without source code or vulnerability information, and must combine static and dynamic analysis to recover and exploit the vulnerability. - [Supporting source](https://arxiv.org/abs/2611.10723) ## Source-stated CyberBinGym contains 100 tasks based on vulnerabilities in 53 widely used projects. > a benchmark containing 100 tasks derived from real vulnerabilities in 53 widely used projects - [Supporting source](https://arxiv.org/abs/2611.10723) ## Source-stated The strongest evaluated agent recovered the target vulnerability in 88% of tasks and exploited it in 20%. > The strongest evaluated agent recovers the target vulnerability in 88% of tasks and exploits it in 20%. - [Supporting source](https://arxiv.org/abs/2611.10723) ## Source-stated On a matched 20-task subset, providing the patched binary raised the agent’s exploit rate from 10% to 25%; with standard binary hardening, it exploited two tasks on which it had previously succeeded. > Providing the patched binary raises its exploit rate from 10% to 25% on a matched 20-task subset; with standard binary hardening enabled, it still exploits two of its 20 previously successful tasks. - [Supporting source](https://arxiv.org/abs/2611.10723) ## Source-stated The authors say these capabilities could lower barriers to targeting unpatched software and shorten defenders’ effective response window, while noting exploitation remains unreliable. > These capabilities lower the barriers to targeting unpatched software and compress defenders' effective response window, increasing the urgency of timely patch deployment. - [Supporting source](https://arxiv.org/abs/2611.10723) ## Source and contributor context Wijayadi, Lambang Akbar, Xiao, Zhaoqi, Wang, Zhun, Li, Hongwei, Guo, Wenbo, Yin, Heng, Liang, Zhenkai, Song, Dawn · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## AI agent An AI system that chooses and carries out actions toward a goal, often using software tools. - [Definition reference](https://www.ibm.com/think/topics/ai-agents) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 87b5801e-7ff3-49f8-8964-5b6970dac6e5. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-12T10:18:04.988Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 9816c70c9e9fad1741ac453e1686b3a641c337f171f3e77ee1acd98348d683a6. Source text hash: 2e44b598a7114e2d87e5b1e052b18af8b39a8e3fbe9e98134296dfc0d1d5f31a. ## Revision and source-check history ### Version 1 — 2026-11-12T10:18:04.988Z The authors of CyberBinGym argue that comparing a released patch with an unpatched binary can help AI agents find and exploit vulnerabilities before systems are updated. They introduce a benchmark in which agents receive stripped vulnerable and patched binaries, without source code or vulnerability information, and must use static and dynamic analysis to recover and exploit the vulnerability. The benchmark contains 100 tasks based on real vulnerabilities in 53 projects. The strongest evaluated agent recovered the target vulnerability in 88% of tasks but exploited it in 20%, showing a gap between finding a vulnerability and turning it into a working exploit in this controlled setting. On a matched 20-task subset, access to the patched binary raised the agent’s exploit rate from 10% to 25%; with standard binary hardening enabled, it still exploited two tasks it had previously succeeded on. The authors argue that these capabilities lower barriers to targeting unpatched software and compress defenders’ effective response window, increasing the urgency of timely patch deployment. The results concern benchmark tasks, and the authors describe exploitation as unreliable; they do not establish how often agents could exploit systems outside this controlled setting. ## Original source arXiv cs.AI · Article Original source date: Nov 7, 2026 (2026-11-07). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-12T10:18:04.988Z. First observed: 2026-11-10T08:39:41.293Z. Retrieved: 2026-11-12T10:16:39.519Z. - [CyberBinGym: Can AI Agents Turn Binary Patches into Real Exploits?](https://arxiv.org/abs/2611.10723) ## Limits The reported results are from benchmark tasks in a controlled setting; the evidence does not establish how often agents could exploit systems outside that setting. The authors characterize exploitation as unreliable; vulnerability recovery and successful exploitation are measured separately. ## Open question How well would the benchmark results carry over to systems outside the controlled setting? ## Source-stated The authors introduce a benchmark requiring agents to recover and exploit vulnerabilities from stripped vulnerable and patched binaries, without source code or vulnerability information. > Agents receive stripped vulnerable and patched binaries without source code or vulnerability information, and must combine static and dynamic analysis to recover and exploit the vulnerability. - [Supporting source](https://arxiv.org/abs/2611.10723) ## Source-stated CyberBinGym contains 100 tasks based on vulnerabilities in 53 widely used projects. > a benchmark containing 100 tasks derived from real vulnerabilities in 53 widely used projects - [Supporting source](https://arxiv.org/abs/2611.10723) ## Source-stated The strongest evaluated agent recovered the target vulnerability in 88% of tasks and exploited it in 20%. > The strongest evaluated agent recovers the target vulnerability in 88% of tasks and exploits it in 20%. - [Supporting source](https://arxiv.org/abs/2611.10723) ## Source-stated On a matched 20-task subset, providing the patched binary raised the agent’s exploit rate from 10% to 25%; with standard binary hardening, it exploited two tasks on which it had previously succeeded. > Providing the patched binary raises its exploit rate from 10% to 25% on a matched 20-task subset; with standard binary hardening enabled, it still exploits two of its 20 previously successful tasks. - [Supporting source](https://arxiv.org/abs/2611.10723) ## Source-stated The authors say these capabilities could lower barriers to targeting unpatched software and shorten defenders’ effective response window, while noting exploitation remains unreliable. > These capabilities lower the barriers to targeting unpatched software and compress defenders' effective response window, increasing the urgency of timely patch deployment. - [Supporting source](https://arxiv.org/abs/2611.10723) ## Source and contributor context Wijayadi, Lambang Akbar, Xiao, Zhaoqi, Wang, Zhun, Li, Hongwei, Guo, Wenbo, Yin, Heng, Liang, Zhenkai, Song, Dawn · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## AI agent An AI system that chooses and carries out actions toward a goal, often using software tools. - [Definition reference](https://www.ibm.com/think/topics/ai-agents) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 87b5801e-7ff3-49f8-8964-5b6970dac6e5. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-12T10:18:04.988Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 9816c70c9e9fad1741ac453e1686b3a641c337f171f3e77ee1acd98348d683a6. Source text hash: 2e44b598a7114e2d87e5b1e052b18af8b39a8e3fbe9e98134296dfc0d1d5f31a. ### uncertain — 2026-11-12T10:18:04.988Z — anthropic/claude-opus-5.5 Publication: 87b5801e-7ff3-49f8-8964-5b6970dac6e5 attribution — pass: The reading attributes the benchmark, results and risk argument to the paper's authors. The arXiv identity matches the source. Conclusions such as lowered barriers and compressed response windows remain framed as the authors' arguments. > Authors:Lambang Akbar Wijayadi, Zhaoqi Xiao, Zhun Wang, Hongwei Li, Wenbo Guo, Heng Yin, Zhenkai Liang, Dawn Song summary — uncertain: The figures are accurate, but central terms go unexplained for non-specialists: binary (compiled program), stripped binaries, static and dynamic analysis, exploit and binary hardening. Readers may also miss the premise that fixes ship as compiled updates while deployment lags. The hardening sentence drops 'of its 20', which hides how sharply hardening cut success. Smallest repair: briefly gloss binary, stripped and hardening. Then state 'two of its 20 previously successful tasks'. > with standard binary hardening enabled, it still exploits two of its 20 previously successful tasks date — pass: originalPublished is 2026-11-07, which matches the v1 submission date. The later feed date of 2026-11-10 is correctly not used. > \[Submitted on 7 Nov 2026\] support — pass: Each claim matches the abstract. The stated gap between 88% recovery and 20% exploitation is a direct reading of the reported numbers. The limitation that the results apply to a controlled setting with unreliable exploitation is source-stated. Claim 5's 'could lower' slightly softens the authors' assertive wording but does not misrepresent their argument. The question follows from the source's controlled-setting caveat. > current agents can turn released binary patches into working exploits in a controlled setting, although exploitation remains unreliable media — pass: The selected media is the verified direct link to the arXiv abstract page of this paper. It is the most direct way to reach the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/0fe272ccfe84989d7ba1c7f6a190629972b710f92cece2572b42565e2b4168fe/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-cyberbingym-can-ai-agents-turn-binary-patches-into-0fe272cc.json)