Cheng, Sikai, Ye, Tinghan, Deza, Arnaud, Qi, Xiaonan, Li, Miao, Liu, Litong, Guan, Changkun · Oct 28, 2026
arXiv: DecompGym benchmarks LLM agents on decomposition algorithms
Markdown · JSON · Original source
The authors introduce DecompGym, a for testing whether coding agents can design and implement Benders decomposition and column generation solvers for prescribed methods. These algorithms pass cuts, columns, and dual values between a master problem and subproblems to solve large optimization problems. The authors assess solver execution, numerical feasibility, a method audit of the code, and closeness to a stored optimal objective. Across 81 instances from nine problem families, two models solve every instance on the first attempt, while seven solve more than 90%. Escape Meridian’s interpretation: passing output checks alone may not show that a solver followed the required method. The authors report that output checks would accept 93 of 108 attempts rejected by the method audit. They also report that reflection, which diagnoses a failure before another attempt, turns 66% of retries into successes, compared with 39% for regeneration. The authors argue that agents building decomposition solvers need checks on how they reach an answer, as well as feedback that locates errors. Whether these benchmark results transfer to other problems or less prescribed solver-design tasks remains open.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 28, 2026 (2026-10-28).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-11T09:11:15.419Z. First observed: 2026-10-30T06:45:39.209Z. Retrieved: 2026-10-31T09:03:15.548Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The evaluates agents writing solvers for a prescribed method; the evidence does not establish performance on other problem types or less constrained solver-design tasks. The abstract does not provide enough detail to assess how the method audit was designed or how its judgments were validated. The abstract contains corrupted text in the description of reflection and one failure type, so those details are not summarized.
Open question
Do method audits and failure-specific feedback improve solver design beyond DecompGym’s prescribed methods and instances?
Source-stated
The authors introduce DecompGym to evaluate coding agents designing solvers for prescribed decomposition methods.
We introduce DecompGym, 81 instances from nine problem families (five Benders, four column generation), in which an agent writes a solver for a prescribed method.
Source-stated
The benchmark checks execution time, numerical feasibility and method adherence, and proximity to a stored optimal objective.
Each solver must meet three criteria: execution within a 60-second time limit; numerical feasibility together with a method audit, in which an LLM judge reads the solver's code; and an objective within 1% of the stored optimal objective.
Source-stated
On the first attempt, two evaluated models solve every instance and seven solve more than 90%.
On the first attempt, two models solve every instance and seven solve more than 90%.
Escape Meridian interpretation
Output checks alone would accept many attempts that fail the method audit, illustrating that an acceptable result can still fail the required method check.
Output checks alone would accept 93 of the 108 attempts that fail the method audit.
Source-stated
The authors report higher retry success with reflection than with regeneration.
this http URL turns 66% of retries into successes, compared with 39% for regeneration
Source-stated
The authors conclude that decomposition agents need checks on their solution process and feedback that helps identify errors.
Agents for decomposition therefore need checks on how a solver reaches its answer and feedback that locates the error.
Source and contributor context
Cheng, Sikai, Ye, Tinghan, Deza, Arnaud, Qi, Xiaonan, Li, Miao, Liu, Litong, Guan, Changkun · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Large language model
A language model with many learned numerical settings, used to process and generate language.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 3997dd5a-daeb-4e40-8e5f-bfbb25687100.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-11T09:11:15.419Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 63fe74ac160959c85546dbf15b0e6aadf98d6540ce23f2f273e649654fdc004f. Source text hash: 341ad5ff20933c970005730af440b08e32d3456626d1849b4ac11dcfa0a466d8.
Revision and source-check history
Version 1 · 2026-11-11T09:11:15.419Z
arXiv: DecompGym benchmarks LLM agents on decomposition algorithms
The authors introduce DecompGym, a for testing whether coding agents can design and implement Benders decomposition and column generation solvers for prescribed methods. These algorithms pass cuts, columns, and dual values between a master problem and subproblems to solve large optimization problems. The authors assess solver execution, numerical feasibility, a method audit of the code, and closeness to a stored optimal objective. Across 81 instances from nine problem families, two models solve every instance on the first attempt, while seven solve more than 90%. Escape Meridian’s interpretation: passing output checks alone may not show that a solver followed the required method. The authors report that output checks would accept 93 of 108 attempts rejected by the method audit. They also report that reflection, which diagnoses a failure before another attempt, turns 66% of retries into successes, compared with 39% for regeneration. The authors argue that agents building decomposition solvers need checks on how they reach an answer, as well as feedback that locates errors. Whether these benchmark results transfer to other problems or less prescribed solver-design tasks remains open.
Original source
cs.AI · Article
Source details
Original source date: Oct 28, 2026 (2026-10-28).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-11T09:11:15.419Z. First observed: 2026-10-30T06:45:39.209Z. Retrieved: 2026-10-31T09:03:15.548Z.
Limits
The evaluates agents writing solvers for a prescribed method; the evidence does not establish performance on other problem types or less constrained solver-design tasks. The abstract does not provide enough detail to assess how the method audit was designed or how its judgments were validated. The abstract contains corrupted text in the description of reflection and one failure type, so those details are not summarized.
Open question
Do method audits and failure-specific feedback improve solver design beyond DecompGym’s prescribed methods and instances?
Source-stated
The authors introduce DecompGym to evaluate coding agents designing solvers for prescribed decomposition methods.
We introduce DecompGym, 81 instances from nine problem families (five Benders, four column generation), in which an agent writes a solver for a prescribed method.
Source-stated
The benchmark checks execution time, numerical feasibility and method adherence, and proximity to a stored optimal objective.
Each solver must meet three criteria: execution within a 60-second time limit; numerical feasibility together with a method audit, in which an LLM judge reads the solver's code; and an objective within 1% of the stored optimal objective.
Source-stated
On the first attempt, two evaluated models solve every instance and seven solve more than 90%.
On the first attempt, two models solve every instance and seven solve more than 90%.
Escape Meridian interpretation
Output checks alone would accept many attempts that fail the method audit, illustrating that an acceptable result can still fail the required method check.
Output checks alone would accept 93 of the 108 attempts that fail the method audit.
Source-stated
The authors report higher retry success with reflection than with regeneration.
this http URL turns 66% of retries into successes, compared with 39% for regeneration
Source-stated
The authors conclude that decomposition agents need checks on their solution process and feedback that helps identify errors.
Agents for decomposition therefore need checks on how a solver reaches its answer and feedback that locates the error.
Source and contributor context
Cheng, Sikai, Ye, Tinghan, Deza, Arnaud, Qi, Xiaonan, Li, Miao, Liu, Litong, Guan, Changkun · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Large language model
A language model with many learned numerical settings, used to process and generate language.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 3997dd5a-daeb-4e40-8e5f-bfbb25687100.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-11T09:11:15.419Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 63fe74ac160959c85546dbf15b0e6aadf98d6540ce23f2f273e649654fdc004f. Source text hash: 341ad5ff20933c970005730af440b08e32d3456626d1849b4ac11dcfa0a466d8.
uncertain · 2026-11-11T09:11:15.419Z · anthropic/claude-opus-5.5
Publication: 3997dd5a-daeb-4e40-8e5f-bfbb25687100
attribution — pass: The reading attributes the benchmark, results and conclusion to the arXiv paper's authors. It keeps the main conclusion as the authors' argument. It also labels the output-check point as Escape Meridian's interpretation.
Agents for decomposition therefore need checks on how a solver reaches its answer and feedback that locates the error.
summary — uncertain: The summary calls the method audit 'a method audit of the code' but omits that an LLM judge performs it. That condition matters for interpreting the headline 93-of-108 finding, which depends on the judge's verdicts. Smallest repair: say 'a method audit in which an LLM judge reads the solver's code.' The summary could also briefly note that 12 GPT, Claude and Gemini models were tested. Otherwise, the actor, task and significance are clear.
numerical feasibility together with a method audit, in which an LLM judge reads the solver's code
date — pass: originalPublished is 2026-10-28, which matches the v1 submission date on arXiv. The later feed date is correctly not used.
[Submitted on 28 Oct 2026]
support — uncertain: In the source, the sentence giving the 66% figure begins with corrupted text ('this http URL'). Attributing 66% to reflection is a plausible inference, because only reflection and regeneration are compared. However, the reading states it as a plain report. Its own limitation also says the corrupted reflection details are not summarized, which contradicts this. Smallest repair: flag the corrupted subject, or drop that limitation clause. Other claims match the excerpts, and the open question is not leading.
this http URL turns 66% of retries into successes, compared with 39% for regeneration
media — pass: The selected media is the verified arXiv abstract page for this paper, which is the direct original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference