# What Does a Harness Buy? Tokens, Mostly

[Canonical reading](https://escapemeridian.com/readings/what-does-a-harness-buy-tokens-mostly-6d621d61)

Yangze Liu and Zhongyi Han argue that a coding agent’s harness—the system prompt, tools and context management around a language model—does not have a universally best configuration. Their study compares three production harnesses across seven models and three benchmarks, and reruns each setup unchanged to measure how much results vary without changing the harness. The authors report that no harness leads consistently across models. Where the data can bound the difference between two harnesses, the bound is 2 to 5 pass-rate points, and swapping harnesses changes about as many task outcomes as rerunning the same setup. But averages can hide consequential pairings: OpenCode trails by 8 to 9 points on one model and leads by 14 on another. Costs also depend on the model, ranging from under 10% to 3 to 4 times across harnesses. The authors’ practical point is that a harness comparison needs a rerun alongside it: otherwise, a measured difference may be mistaken for a harness effect when reruns themselves change outcomes. Their results make “which harness is better?” a model-dependent question, not a general ranking. The abstract does not identify which model-harness pairings produce the reported score and cost differences.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

## Original source

arXiv cs.AI · Article

Original source date: Oct 3, 2026 (2026-10-03).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-19T08:18:26.903Z. First observed: 2026-10-06T08:47:13.776Z. Retrieved: 2026-11-19T08:16:59.280Z.

- [What Does a Harness Buy? Tokens, Mostly](https://arxiv.org/abs/2610.04433)

## Limits

The abstract reports tests of seven models, three production harnesses and three benchmarks; it does not establish that the findings generalize to other models, harnesses or tasks. The abstract does not identify which model-harness pairings produce the reported score and cost differences.


## Open question

Across the tested model-harness pairings, how often do harness differences exceed the variation seen in reruns?


## Source-stated

The authors define a coding agent’s harness as the system prompt, tools and context management surrounding a language model.

> A coding agent is a language model inside a harness: the system prompt, the tools, and the context management

- [Supporting source](https://arxiv.org/abs/2610.04433)

## Source-stated

Across their tested models and benchmarks, the authors report no harness consistently leads, and bounded differences between harnesses are 2 to 5 points.

> No harness leads consistently across models. Where the data can bound the difference between two harnesses, the bound is 2 to 5 points

- [Supporting source](https://arxiv.org/abs/2610.04433)

## Source-stated

The authors report that changing the harness alters about as many task outcomes as rerunning the same harness, with some model-specific reversals.

> a harness swap changes about as many task outcomes as a rerun. The exceptions are harness effects in particular pairings of model and harness, whose direction changes with the model

- [Supporting source](https://arxiv.org/abs/2610.04433)

## Source-stated

The authors report that cost differences across harnesses depend on the model, ranging from under 10% to 3 to 4 times.

> Cost differences between harnesses also depend on the model, from under 10% to 3 to 4 times.

- [Supporting source](https://arxiv.org/abs/2610.04433)

## Source and contributor context

Liu, Yangze, Han, Zhongyi · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

- [Definition reference](https://arxiv.org/abs/2211.09110)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: 31975bf2-91f5-475b-8f10-4b080414f245.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-19T08:18:26.903Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: e0d321ea118b6ea45b020f4c7e4e7358c7478fe1b95b9ad818cd436e26dcb50b. Source text hash: 4e16fae4c939559b7bc6f3d7046c48f8af0205c8b04f7f1bdcd0e6dc9b1fffae.


## Revision and source-check history

### Version 1 — 2026-11-19T08:18:26.903Z

Yangze Liu and Zhongyi Han argue that a coding agent’s harness—the system prompt, tools and context management around a language model—does not have a universally best configuration. Their study compares three production harnesses across seven models and three benchmarks, and reruns each setup unchanged to measure how much results vary without changing the harness. The authors report that no harness leads consistently across models. Where the data can bound the difference between two harnesses, the bound is 2 to 5 pass-rate points, and swapping harnesses changes about as many task outcomes as rerunning the same setup. But averages can hide consequential pairings: OpenCode trails by 8 to 9 points on one model and leads by 14 on another. Costs also depend on the model, ranging from under 10% to 3 to 4 times across harnesses. The authors’ practical point is that a harness comparison needs a rerun alongside it: otherwise, a measured difference may be mistaken for a harness effect when reruns themselves change outcomes. Their results make “which harness is better?” a model-dependent question, not a general ranking. The abstract does not identify which model-harness pairings produce the reported score and cost differences.

## Original source

arXiv cs.AI · Article

Original source date: Oct 3, 2026 (2026-10-03).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-19T08:18:26.903Z. First observed: 2026-10-06T08:47:13.776Z. Retrieved: 2026-11-19T08:16:59.280Z.

- [What Does a Harness Buy? Tokens, Mostly](https://arxiv.org/abs/2610.04433)

## Limits

The abstract reports tests of seven models, three production harnesses and three benchmarks; it does not establish that the findings generalize to other models, harnesses or tasks. The abstract does not identify which model-harness pairings produce the reported score and cost differences.


## Open question

Across the tested model-harness pairings, how often do harness differences exceed the variation seen in reruns?


## Source-stated

The authors define a coding agent’s harness as the system prompt, tools and context management surrounding a language model.

> A coding agent is a language model inside a harness: the system prompt, the tools, and the context management

- [Supporting source](https://arxiv.org/abs/2610.04433)

## Source-stated

Across their tested models and benchmarks, the authors report no harness consistently leads, and bounded differences between harnesses are 2 to 5 points.

> No harness leads consistently across models. Where the data can bound the difference between two harnesses, the bound is 2 to 5 points

- [Supporting source](https://arxiv.org/abs/2610.04433)

## Source-stated

The authors report that changing the harness alters about as many task outcomes as rerunning the same harness, with some model-specific reversals.

> a harness swap changes about as many task outcomes as a rerun. The exceptions are harness effects in particular pairings of model and harness, whose direction changes with the model

- [Supporting source](https://arxiv.org/abs/2610.04433)

## Source-stated

The authors report that cost differences across harnesses depend on the model, ranging from under 10% to 3 to 4 times.

> Cost differences between harnesses also depend on the model, from under 10% to 3 to 4 times.

- [Supporting source](https://arxiv.org/abs/2610.04433)

## Source and contributor context

Liu, Yangze, Han, Zhongyi · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

- [Definition reference](https://arxiv.org/abs/2211.09110)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: 31975bf2-91f5-475b-8f10-4b080414f245.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-19T08:18:26.903Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: e0d321ea118b6ea45b020f4c7e4e7358c7478fe1b95b9ad818cd436e26dcb50b. Source text hash: 4e16fae4c939559b7bc6f3d7046c48f8af0205c8b04f7f1bdcd0e6dc9b1fffae.


### passed — 2026-11-19T08:18:26.903Z — anthropic/claude-opus-5.5

Publication: 31975bf2-91f5-475b-8f10-4b080414f245

attribution — pass: The reading correctly credits the paper to Yangze Liu and Zhongyi Han. It frames the findings as the authors' reports and arguments, not as established fact.

> Authors:Yangze Liu, Zhongyi Han

summary — pass: The summary defines harness in plain terms and covers the design: seven models, three production harnesses, three benchmarks and unchanged reruns. It also covers the main findings, the OpenCode reversal, the cost spread and the reporting recommendation. The rerun rationale fairly reflects the source's stated gap. Optional repair: name all three harnesses and note that vendors and leaderboards credit harnesses with score gains. That claim is the stake the paper challenges. Neither omission blocks understanding. The question about how often harness differences exceed rerun variation follows from the evidence.

> have not measured whether the swap changes the pass rate more than running the same harness again

date — pass: The originalPublished date of 2026-10-03 matches the v1 submission date. The source was later revised on 18 Nov 2026, but the first submission correctly governs.

> Submitted on 3 Oct 2026 (v1)

support — pass: Each claim is an exact or faithful paraphrase of the abstract: the harness definition, the 2 to 5 point bound, a swap matching a rerun, the model-specific reversals and the cost range. The limitations are accurate and appropriately scoped. The reading does not overstate generality or turn benchmark results into real-world reliability.

> Which harness is better therefore has no answer apart from the model, and a harness difference should be reported next to a rerun.

media — pass: The selected media is the verified direct arXiv abstract page for this paper, which gives the easiest access to the actual source.

## Recorded references

Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter.

No public references recorded here yet.

[Reference data (JSON)](/readings/6d621d61ee5a890844f2d0583daddb27cfa89448fd63577b5ea4b3c0d103b2f4/references.json)

## Corrections and challenges

[Send a correction or an opposing source](https://escapemeridian.com/corrections/)

[JSON record](https://escapemeridian.com/readings/what-does-a-harness-buy-tokens-mostly-6d621d61.json)
