Cheng, Bobber · Nov 9, 2026
Bobber Cheng on compile-gated Lean proof refactoring
Markdown · JSON · Original source
Bobber Cheng reports that his team’s closed-track entry in the Refactor Arena scored 86.31 points across the final problems, for US$166.33 at list prices. He describes three practices. The team chose a model for each paid stage based on its own cost and performance tests; turned lessons from development and research runs into compiler-checked programmatic rewrite rules; and recorded and reviewed every call before each change. The rewrite rules fixed before release added points, while gains from later passes were measured in sample. Cheng also reports that nine board scores were close to predictions from the team’s meter. These are results from the team’s competition entry, not a demonstration of performance in other settings. The abstract says the full report includes the bill and the spread of three runs, but does not give the run-to-run spread.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 9, 2026 (2026-11-09).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-11T21:32:23.588Z. First observed: 2026-11-11T05:09:27.974Z. Retrieved: 2026-11-11T21:04:39.573Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The evidence is the paper’s abstract and describes a closed-track competition entry; it does not establish performance in other settings. The abstract mentions the spread of three runs but does not provide the run-to-run values or explain variation.
Open question
How much did the entry’s results vary across the three runs, and what does the report identify as explaining that variation?
Source-stated
Cheng reports that the team’s closed-track entry scored 86.31 points across the final problems, at a total list-price cost of US$166.33.
scored 86.31 points on the final board (L = 77.07, H = 81.86, Z = 100) over the 60 final problems
Source-stated
The team chose models for paid stages based on which of nine tested models performed best per dollar in its own harness.
each paid stage runs the model that measured best per dollar for it among the 9 we tested in our own harness
Source-stated
The team reports that programmatic rewrite rules fixed before release added 0.93 points; gains from later passes were measured in sample.
+0.93 points from rules fixed before the release; +0.26 and +0.30, in sample, from passes built or switched on after the release
Source-stated
The team says its nine board scores were close to predictions from its meter.
all 9 board scores came within -0.063 to +0.015 points of our meter's prediction
Source-stated
The abstract says the report includes the full bill and the spread of three runs, but does not give the spread values.
We report the full bill and the spread of three runs.
Source and contributor context
Cheng, Bobber · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Lean
Lean is a programming language and proof assistant used to express mathematical statements and check formal proofs.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 8d5992d6-f0ba-4b4b-8d63-36a3ea7b8123.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-11T21:32:23.588Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: d1c5dbc13378089dbd7d4bb73b890218fd6ca59de6e3023f65604dd5dcd76795. Source text hash: 7493f29bda68763ef1285d93e126d1d1023e6d540e6d35f1b4cb10c3cba47af1.
Revision and source-check history
Version 1 · 2026-11-11T21:32:23.588Z
Bobber Cheng on compile-gated Lean proof refactoring
Bobber Cheng reports that his team’s closed-track entry in the Refactor Arena scored 86.31 points across the final problems, for US$166.33 at list prices. He describes three practices. The team chose a model for each paid stage based on its own cost and performance tests; turned lessons from development and research runs into compiler-checked programmatic rewrite rules; and recorded and reviewed every call before each change. The rewrite rules fixed before release added points, while gains from later passes were measured in sample. Cheng also reports that nine board scores were close to predictions from the team’s meter. These are results from the team’s competition entry, not a demonstration of performance in other settings. The abstract says the full report includes the bill and the spread of three runs, but does not give the run-to-run spread.
Original source
cs.AI · Article
Source details
Original source date: Nov 9, 2026 (2026-11-09).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-11T21:32:23.588Z. First observed: 2026-11-11T05:09:27.974Z. Retrieved: 2026-11-11T21:04:39.573Z.
Limits
The evidence is the paper’s abstract and describes a closed-track competition entry; it does not establish performance in other settings. The abstract mentions the spread of three runs but does not provide the run-to-run values or explain variation.
Open question
How much did the entry’s results vary across the three runs, and what does the report identify as explaining that variation?
Source-stated
Cheng reports that the team’s closed-track entry scored 86.31 points across the final problems, at a total list-price cost of US$166.33.
scored 86.31 points on the final board (L = 77.07, H = 81.86, Z = 100) over the 60 final problems
Source-stated
The team chose models for paid stages based on which of nine tested models performed best per dollar in its own harness.
each paid stage runs the model that measured best per dollar for it among the 9 we tested in our own harness
Source-stated
The team reports that programmatic rewrite rules fixed before release added 0.93 points; gains from later passes were measured in sample.
+0.93 points from rules fixed before the release; +0.26 and +0.30, in sample, from passes built or switched on after the release
Source-stated
The team says its nine board scores were close to predictions from its meter.
all 9 board scores came within -0.063 to +0.015 points of our meter's prediction
Source-stated
The abstract says the report includes the full bill and the spread of three runs, but does not give the spread values.
We report the full bill and the spread of three runs.
Source and contributor context
Cheng, Bobber · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Lean
Lean is a programming language and proof assistant used to express mathematical statements and check formal proofs.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 8d5992d6-f0ba-4b4b-8d63-36a3ea7b8123.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-11T21:32:23.588Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: d1c5dbc13378089dbd7d4bb73b890218fd6ca59de6e3023f65604dd5dcd76795. Source text hash: 7493f29bda68763ef1285d93e126d1d1023e6d540e6d35f1b4cb10c3cba47af1.
uncertain · 2026-11-11T21:32:23.588Z · anthropic/claude-opus-5.5
Publication: 8d5992d6-f0ba-4b4b-8d63-36a3ea7b8123
attribution — uncertain: The work is correctly attributed to Bobber Cheng and team Vexorium-Agent, but 'his team' assigns a gender the source does not state. Smallest repair: 'Bobber Cheng reports that the team's closed-track entry...' or name team Vexorium-Agent.
Technical report of team Vexorium-Agent for the closed track of the Lean Refactor Arena
summary — uncertain: Central terms are not explained: Lean, proof refactoring, closed track, compile-gated and in sample. A general reader cannot tell what was improved or why it matters. The reading also omits context that gives the 86.31 score meaning: reference proofs scored 33.33, the entry placed 8th against a top score of 92.57, and there was a US$3-per-problem cap. Suggested term-library tooltips: Lean, refactoring, compile-gated. Add one sentence with this comparison and the cap.
against 33.33 for the reference proofs, 8th at the deadline (top: 92.57), for US\$166.33 at list prices with every call counted (per problem: mean US\$2.772, at most US\$2.874; US\$3 cap)
date — pass: originalPublished 2026-11-09 matches the v1 submission date. The feed date of 2026-11-11 is later and correctly not used.
[Submitted on 9 Nov 2026]
support — pass: The claims are stated in the abstract and are kept attributed. The claim about the US$166.33 cost uses an excerpt that omits the cost figure, although the abstract elsewhere states it. The caveat about generalization is a fair limitation. The question about the spread of the three runs follows from the abstract's unreported values and does not assert an answer.
We report the full bill and the spread of three runs.
media — pass: The selected media is the verified direct link to the arXiv abstract page, which is the original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
Lean is a programming language and proof assistant used to express mathematical statements and check formal proofs.
Explore LeanarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivLean is a programming language and proof assistant used to express mathematical statements and check formal proofs.
Explore LeanarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv