# arXiv: DecompGym benchmarks LLM agents on decomposition algorithms [Canonical reading](https://escapemeridian.com/readings/arxiv-decompgym-benchmarks-llm-agents-on-decomposition-a8eda828) The authors introduce DecompGym, a benchmark for testing whether coding agents can design and implement Benders decomposition and column generation solvers for prescribed methods. These algorithms pass cuts, columns, and dual values between a master problem and subproblems to solve large optimization problems. The authors assess solver execution, numerical feasibility, a method audit of the code, and closeness to a stored optimal objective. Across 81 instances from nine problem families, two models solve every instance on the first attempt, while seven solve more than 90%. Escape Meridian’s interpretation: passing output checks alone may not show that a solver followed the required method. The authors report that output checks would accept 93 of 108 attempts rejected by the method audit. They also report that reflection, which diagnoses a failure before another attempt, turns 66% of retries into successes, compared with 39% for regeneration. The authors argue that agents building decomposition solvers need checks on how they reach an answer, as well as feedback that locates errors. Whether these benchmark results transfer to other problems or less prescribed solver-design tasks remains open. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Oct 28, 2026 (2026-10-28). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-11T09:11:15.419Z. First observed: 2026-10-30T06:45:39.209Z. Retrieved: 2026-10-31T09:03:15.548Z. - [DecompGym: Benchmarking LLM Agents on Designing Decomposition Algorithms](https://arxiv.org/abs/2610.38175) ## Limits The benchmark evaluates agents writing solvers for a prescribed method; the evidence does not establish performance on other problem types or less constrained solver-design tasks. The abstract does not provide enough detail to assess how the method audit was designed or how its judgments were validated. The abstract contains corrupted text in the description of reflection and one failure type, so those details are not summarized. ## Open question Do method audits and failure-specific feedback improve solver design beyond DecompGym’s prescribed methods and benchmark instances? ## Source-stated The authors introduce DecompGym to evaluate coding agents designing solvers for prescribed decomposition methods. > We introduce DecompGym, 81 instances from nine problem families (five Benders, four column generation), in which an agent writes a solver for a prescribed method. - [Supporting source](https://arxiv.org/abs/2610.38175) ## Source-stated The benchmark checks execution time, numerical feasibility and method adherence, and proximity to a stored optimal objective. > Each solver must meet three criteria: execution within a 60-second time limit; numerical feasibility together with a method audit, in which an LLM judge reads the solver's code; and an objective within 1% of the stored optimal objective. - [Supporting source](https://arxiv.org/abs/2610.38175) ## Source-stated On the first attempt, two evaluated models solve every instance and seven solve more than 90%. > On the first attempt, two models solve every instance and seven solve more than 90%. - [Supporting source](https://arxiv.org/abs/2610.38175) ## Escape Meridian interpretation Output checks alone would accept many attempts that fail the method audit, illustrating that an acceptable result can still fail the required method check. > Output checks alone would accept 93 of the 108 attempts that fail the method audit. - [Supporting source](https://arxiv.org/abs/2610.38175) ## Source-stated The authors report higher retry success with reflection than with regeneration. > this http URL turns 66% of retries into successes, compared with 39% for regeneration - [Supporting source](https://arxiv.org/abs/2610.38175) ## Source-stated The authors conclude that decomposition agents need checks on their solution process and feedback that helps identify errors. > Agents for decomposition therefore need checks on how a solver reaches its answer and feedback that locates the error. - [Supporting source](https://arxiv.org/abs/2610.38175) ## Source and contributor context Cheng, Sikai, Ye, Tinghan, Deza, Arnaud, Qi, Xiaonan, Li, Miao, Liu, Litong, Guan, Changkun · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 3997dd5a-daeb-4e40-8e5f-bfbb25687100. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-11T09:11:15.419Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 63fe74ac160959c85546dbf15b0e6aadf98d6540ce23f2f273e649654fdc004f. Source text hash: 341ad5ff20933c970005730af440b08e32d3456626d1849b4ac11dcfa0a466d8. ## Revision and source-check history ### Version 1 — 2026-11-11T09:11:15.419Z The authors introduce DecompGym, a benchmark for testing whether coding agents can design and implement Benders decomposition and column generation solvers for prescribed methods. These algorithms pass cuts, columns, and dual values between a master problem and subproblems to solve large optimization problems. The authors assess solver execution, numerical feasibility, a method audit of the code, and closeness to a stored optimal objective. Across 81 instances from nine problem families, two models solve every instance on the first attempt, while seven solve more than 90%. Escape Meridian’s interpretation: passing output checks alone may not show that a solver followed the required method. The authors report that output checks would accept 93 of 108 attempts rejected by the method audit. They also report that reflection, which diagnoses a failure before another attempt, turns 66% of retries into successes, compared with 39% for regeneration. The authors argue that agents building decomposition solvers need checks on how they reach an answer, as well as feedback that locates errors. Whether these benchmark results transfer to other problems or less prescribed solver-design tasks remains open. ## Original source arXiv cs.AI · Article Original source date: Oct 28, 2026 (2026-10-28). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-11T09:11:15.419Z. First observed: 2026-10-30T06:45:39.209Z. Retrieved: 2026-10-31T09:03:15.548Z. - [DecompGym: Benchmarking LLM Agents on Designing Decomposition Algorithms](https://arxiv.org/abs/2610.38175) ## Limits The benchmark evaluates agents writing solvers for a prescribed method; the evidence does not establish performance on other problem types or less constrained solver-design tasks. The abstract does not provide enough detail to assess how the method audit was designed or how its judgments were validated. The abstract contains corrupted text in the description of reflection and one failure type, so those details are not summarized. ## Open question Do method audits and failure-specific feedback improve solver design beyond DecompGym’s prescribed methods and benchmark instances? ## Source-stated The authors introduce DecompGym to evaluate coding agents designing solvers for prescribed decomposition methods. > We introduce DecompGym, 81 instances from nine problem families (five Benders, four column generation), in which an agent writes a solver for a prescribed method. - [Supporting source](https://arxiv.org/abs/2610.38175) ## Source-stated The benchmark checks execution time, numerical feasibility and method adherence, and proximity to a stored optimal objective. > Each solver must meet three criteria: execution within a 60-second time limit; numerical feasibility together with a method audit, in which an LLM judge reads the solver's code; and an objective within 1% of the stored optimal objective. - [Supporting source](https://arxiv.org/abs/2610.38175) ## Source-stated On the first attempt, two evaluated models solve every instance and seven solve more than 90%. > On the first attempt, two models solve every instance and seven solve more than 90%. - [Supporting source](https://arxiv.org/abs/2610.38175) ## Escape Meridian interpretation Output checks alone would accept many attempts that fail the method audit, illustrating that an acceptable result can still fail the required method check. > Output checks alone would accept 93 of the 108 attempts that fail the method audit. - [Supporting source](https://arxiv.org/abs/2610.38175) ## Source-stated The authors report higher retry success with reflection than with regeneration. > this http URL turns 66% of retries into successes, compared with 39% for regeneration - [Supporting source](https://arxiv.org/abs/2610.38175) ## Source-stated The authors conclude that decomposition agents need checks on their solution process and feedback that helps identify errors. > Agents for decomposition therefore need checks on how a solver reaches its answer and feedback that locates the error. - [Supporting source](https://arxiv.org/abs/2610.38175) ## Source and contributor context Cheng, Sikai, Ye, Tinghan, Deza, Arnaud, Qi, Xiaonan, Li, Miao, Liu, Litong, Guan, Changkun · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 3997dd5a-daeb-4e40-8e5f-bfbb25687100. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-11T09:11:15.419Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 63fe74ac160959c85546dbf15b0e6aadf98d6540ce23f2f273e649654fdc004f. Source text hash: 341ad5ff20933c970005730af440b08e32d3456626d1849b4ac11dcfa0a466d8. ### uncertain — 2026-11-11T09:11:15.419Z — anthropic/claude-opus-5.5 Publication: 3997dd5a-daeb-4e40-8e5f-bfbb25687100 attribution — pass: The reading attributes the benchmark, results and conclusion to the arXiv paper's authors. It keeps the main conclusion as the authors' argument. It also labels the output-check point as Escape Meridian's interpretation. > Agents for decomposition therefore need checks on how a solver reaches its answer and feedback that locates the error. summary — uncertain: The summary calls the method audit 'a method audit of the code' but omits that an LLM judge performs it. That condition matters for interpreting the headline 93-of-108 finding, which depends on the judge's verdicts. Smallest repair: say 'a method audit in which an LLM judge reads the solver's code.' The summary could also briefly note that 12 GPT, Claude and Gemini models were tested. Otherwise, the actor, task and significance are clear. > numerical feasibility together with a method audit, in which an LLM judge reads the solver's code date — pass: originalPublished is 2026-10-28, which matches the v1 submission date on arXiv. The later feed date is correctly not used. > \[Submitted on 28 Oct 2026\] support — uncertain: In the source, the sentence giving the 66% figure begins with corrupted text ('this http URL'). Attributing 66% to reflection is a plausible inference, because only reflection and regeneration are compared. However, the reading states it as a plain report. Its own limitation also says the corrupted reflection details are not summarized, which contradicts this. Smallest repair: flag the corrupted subject, or drop that limitation clause. Other claims match the excerpts, and the open question is not leading. > this http URL turns 66% of retries into successes, compared with 39% for regeneration media — pass: The selected media is the verified arXiv abstract page for this paper, which is the direct original source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/a8eda8285fe44dd321b6e19f8692a39a0eb83cf1377838598737c555f4920c0a/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-decompgym-benchmarks-llm-agents-on-decomposition-a8eda828.json)