Liu, Yangze, Han, Zhongyi · Oct 3, 2026
What Does a Harness Buy? Tokens, Mostly
Markdown · JSON · Original source
Yangze Liu and Zhongyi Han argue that a coding agent’s harness—the system prompt, tools and context management around a language model—does not have a universally best configuration. Their study compares three production harnesses across seven models and three , and reruns each setup unchanged to measure how much results vary without changing the harness. The authors report that no harness leads consistently across models. Where the data can bound the difference between two harnesses, the bound is 2 to 5 pass-rate points, and swapping harnesses changes about as many task outcomes as rerunning the same setup. But averages can hide consequential pairings: OpenCode trails by 8 to 9 points on one model and leads by 14 on another. Costs also depend on the model, ranging from under 10% to 3 to 4 times across harnesses. The authors’ practical point is that a harness comparison needs a rerun alongside it: otherwise, a measured difference may be mistaken for a harness effect when reruns themselves change outcomes. Their results make “which harness is better?” a model-dependent question, not a general ranking. The abstract does not identify which model-harness pairings produce the reported score and cost differences.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 3, 2026 (2026-10-03).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-19T08:18:26.903Z. First observed: 2026-10-06T08:47:13.776Z. Retrieved: 2026-11-19T08:16:59.280Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The abstract reports tests of seven models, three production harnesses and three ; it does not establish that the findings generalize to other models, harnesses or tasks. The abstract does not identify which model-harness pairings produce the reported score and cost differences.
Open question
Across the tested model-harness pairings, how often do harness differences exceed the variation seen in reruns?
Source-stated
The authors define a coding agent’s harness as the system prompt, tools and context management surrounding a language model.
A coding agent is a language model inside a harness: the system prompt, the tools, and the context management
Source-stated
Across their tested models and benchmarks, the authors report no harness consistently leads, and bounded differences between harnesses are 2 to 5 points.
No harness leads consistently across models. Where the data can bound the difference between two harnesses, the bound is 2 to 5 points
Source-stated
The authors report that changing the harness alters about as many task outcomes as rerunning the same harness, with some model-specific reversals.
a harness swap changes about as many task outcomes as a rerun. The exceptions are harness effects in particular pairings of model and harness, whose direction changes with the model
Source-stated
The authors report that cost differences across harnesses depend on the model, ranging from under 10% to 3 to 4 times.
Cost differences between harnesses also depend on the model, from under 10% to 3 to 4 times.
Source and contributor context
Liu, Yangze, Han, Zhongyi · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 31975bf2-91f5-475b-8f10-4b080414f245.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-19T08:18:26.903Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: e0d321ea118b6ea45b020f4c7e4e7358c7478fe1b95b9ad818cd436e26dcb50b. Source text hash: 4e16fae4c939559b7bc6f3d7046c48f8af0205c8b04f7f1bdcd0e6dc9b1fffae.
Revision and source-check history
Version 1 · 2026-11-19T08:18:26.903Z
What Does a Harness Buy? Tokens, Mostly
Yangze Liu and Zhongyi Han argue that a coding agent’s harness—the system prompt, tools and context management around a language model—does not have a universally best configuration. Their study compares three production harnesses across seven models and three , and reruns each setup unchanged to measure how much results vary without changing the harness. The authors report that no harness leads consistently across models. Where the data can bound the difference between two harnesses, the bound is 2 to 5 pass-rate points, and swapping harnesses changes about as many task outcomes as rerunning the same setup. But averages can hide consequential pairings: OpenCode trails by 8 to 9 points on one model and leads by 14 on another. Costs also depend on the model, ranging from under 10% to 3 to 4 times across harnesses. The authors’ practical point is that a harness comparison needs a rerun alongside it: otherwise, a measured difference may be mistaken for a harness effect when reruns themselves change outcomes. Their results make “which harness is better?” a model-dependent question, not a general ranking. The abstract does not identify which model-harness pairings produce the reported score and cost differences.
Original source
cs.AI · Article
Source details
Original source date: Oct 3, 2026 (2026-10-03).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-19T08:18:26.903Z. First observed: 2026-10-06T08:47:13.776Z. Retrieved: 2026-11-19T08:16:59.280Z.
Limits
The abstract reports tests of seven models, three production harnesses and three ; it does not establish that the findings generalize to other models, harnesses or tasks. The abstract does not identify which model-harness pairings produce the reported score and cost differences.
Open question
Across the tested model-harness pairings, how often do harness differences exceed the variation seen in reruns?
Source-stated
The authors define a coding agent’s harness as the system prompt, tools and context management surrounding a language model.
A coding agent is a language model inside a harness: the system prompt, the tools, and the context management
Source-stated
Across their tested models and benchmarks, the authors report no harness consistently leads, and bounded differences between harnesses are 2 to 5 points.
No harness leads consistently across models. Where the data can bound the difference between two harnesses, the bound is 2 to 5 points
Source-stated
The authors report that changing the harness alters about as many task outcomes as rerunning the same harness, with some model-specific reversals.
a harness swap changes about as many task outcomes as a rerun. The exceptions are harness effects in particular pairings of model and harness, whose direction changes with the model
Source-stated
The authors report that cost differences across harnesses depend on the model, ranging from under 10% to 3 to 4 times.
Cost differences between harnesses also depend on the model, from under 10% to 3 to 4 times.
Source and contributor context
Liu, Yangze, Han, Zhongyi · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 31975bf2-91f5-475b-8f10-4b080414f245.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-19T08:18:26.903Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: e0d321ea118b6ea45b020f4c7e4e7358c7478fe1b95b9ad818cd436e26dcb50b. Source text hash: 4e16fae4c939559b7bc6f3d7046c48f8af0205c8b04f7f1bdcd0e6dc9b1fffae.
passed · 2026-11-19T08:18:26.903Z · anthropic/claude-opus-5.5
Publication: 31975bf2-91f5-475b-8f10-4b080414f245
attribution — pass: The reading correctly credits the paper to Yangze Liu and Zhongyi Han. It frames the findings as the authors' reports and arguments, not as established fact.
Authors:Yangze Liu, Zhongyi Han
summary — pass: The summary defines harness in plain terms and covers the design: seven models, three production harnesses, three benchmarks and unchanged reruns. It also covers the main findings, the OpenCode reversal, the cost spread and the reporting recommendation. The rerun rationale fairly reflects the source's stated gap. Optional repair: name all three harnesses and note that vendors and leaderboards credit harnesses with score gains. That claim is the stake the paper challenges. Neither omission blocks understanding. The question about how often harness differences exceed rerun variation follows from the evidence.
have not measured whether the swap changes the pass rate more than running the same harness again
date — pass: The originalPublished date of 2026-10-03 matches the v1 submission date. The source was later revised on 18 Nov 2026, but the first submission correctly governs.
Submitted on 3 Oct 2026 (v1)
support — pass: Each claim is an exact or faithful paraphrase of the abstract: the harness definition, the 2 to 5 point bound, a swap matching a rerun, the model-specific reversals and the cost range. The limitations are accurate and appropriately scoped. The reading does not overstate generality or turn benchmark results into real-world reliability.
Which harness is better therefore has no answer apart from the model, and a harness difference should be reported next to a rerun.
media — pass: The selected media is the verified direct arXiv abstract page for this paper, which gives the easiest access to the actual source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference