# Can Coding Agents Write and Transpile Unit Tests

[Canonical reading](https://escapemeridian.com/readings/can-coding-agents-write-and-transpile-unit-tests-c4b349a4)

Authors Gil Pasternak, AJ Langley, Tom Yotam and Ellie Kitanidis examine whether coding agents can write tests and translate them between programming languages. They focus on a source of trust in coding assistants: passing tests, which agents often write themselves. In a benchmark of three coding agents, each using a frontier model with high reasoning effort, the authors report that agent-written tests exceeded human-written suites on traditional coverage measures. They also report that this advantage was uneven: test quality fell sharply as project size grew, and much of the coverage advantage came from writing more tests. Agentic test translation looked strong in aggregate but remained inconsistent and unreliable, they say. Escape Meridian interpretation: these findings complicate reliance on passing tests as a sign of confidence when agents write the tests being used to judge their work. The abstract does not provide project sizes, individual results or detailed metrics, limiting how fully readers can assess the reported effects.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

## Original source

arXiv cs.AI · Article

Original source date: Nov 16, 2026 (2026-11-16).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-18T14:48:58.699Z. First observed: 2026-11-18T05:13:48.565Z. Retrieved: 2026-11-18T14:48:03.374Z.

- [Can Coding Agents Write and Transpile Unit Tests](https://arxiv.org/abs/2611.21797)

## Limits

The supplied evidence is the paper’s abstract and does not give project sizes, individual results or detailed metrics for assessing the reported effects. The abstract does not specify how the authors measured test quality or test translation.


## Open question

How did the authors measure test quality and test translation, and how did those results vary across project sizes?


## Source-stated

The authors benchmarked three coding agents, each using a frontier model with high reasoning effort.

> We benchmark three popular coding agents, each evaluated with a native frontier model and high reasoning effort.

- [Supporting source](https://arxiv.org/abs/2611.21797)

## Source-stated

The authors report that agent-written test suites exceeded human-written suites on traditional coverage measures.

> coding agents far exceed human written suites on traditional measures of coverage.

- [Supporting source](https://arxiv.org/abs/2611.21797)

## Source-stated

The authors say test quality declined sharply with project size, and that more tests accounted for much of the coverage advantage.

> agent test quality degrades sharply with project size, and much of the advantage lies in a greater number of written tests.

- [Supporting source](https://arxiv.org/abs/2611.21797)

## Source-stated

The authors characterize test translation by agents as inconsistent and unreliable despite strong aggregate results.

> agentic test translation---while seemingly strong in aggregate---remains inconsistent and unreliable.

- [Supporting source](https://arxiv.org/abs/2611.21797)

## Escape Meridian interpretation

Passing tests may not establish confidence in agent-written code when the agents also write the tests; this is an interpretation of the authors’ stated concern and findings.

> Much of this trust rests on passing tests, which are often written by the coding agents themselves.

- [Supporting source](https://arxiv.org/abs/2611.21797)

## Source and contributor context

Pasternak, Gil, Langley, AJ, Yotam, Tom, Kitanidis, Ellie · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

- [Definition reference](https://arxiv.org/abs/2211.09110)

## Frontier model

One of the most capable AI models available at a given time; the set shifts as models improve.

- [Definition reference](https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper)

## Agentic AI

AI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer.

- [Definition reference](https://developers.google.com/machine-learning/glossary#agentic)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: 8f0bd417-2c81-42d6-8266-ae737ae33948.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-18T14:48:58.699Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 91391d95b49b50a272f3cb1e250f25ea4dabf5712066cf337fa726761cbadd2c. Source text hash: 2ff3cd06859baae4317c1aaaf574ea7fb3cac08f5b79c8cf821e83fc2784f911.


## Revision and source-check history

### Version 1 — 2026-11-18T14:48:58.699Z

Authors Gil Pasternak, AJ Langley, Tom Yotam and Ellie Kitanidis examine whether coding agents can write tests and translate them between programming languages. They focus on a source of trust in coding assistants: passing tests, which agents often write themselves. In a benchmark of three coding agents, each using a frontier model with high reasoning effort, the authors report that agent-written tests exceeded human-written suites on traditional coverage measures. They also report that this advantage was uneven: test quality fell sharply as project size grew, and much of the coverage advantage came from writing more tests. Agentic test translation looked strong in aggregate but remained inconsistent and unreliable, they say. Escape Meridian interpretation: these findings complicate reliance on passing tests as a sign of confidence when agents write the tests being used to judge their work. The abstract does not provide project sizes, individual results or detailed metrics, limiting how fully readers can assess the reported effects.

## Original source

arXiv cs.AI · Article

Original source date: Nov 16, 2026 (2026-11-16).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-18T14:48:58.699Z. First observed: 2026-11-18T05:13:48.565Z. Retrieved: 2026-11-18T14:48:03.374Z.

- [Can Coding Agents Write and Transpile Unit Tests](https://arxiv.org/abs/2611.21797)

## Limits

The supplied evidence is the paper’s abstract and does not give project sizes, individual results or detailed metrics for assessing the reported effects. The abstract does not specify how the authors measured test quality or test translation.


## Open question

How did the authors measure test quality and test translation, and how did those results vary across project sizes?


## Source-stated

The authors benchmarked three coding agents, each using a frontier model with high reasoning effort.

> We benchmark three popular coding agents, each evaluated with a native frontier model and high reasoning effort.

- [Supporting source](https://arxiv.org/abs/2611.21797)

## Source-stated

The authors report that agent-written test suites exceeded human-written suites on traditional coverage measures.

> coding agents far exceed human written suites on traditional measures of coverage.

- [Supporting source](https://arxiv.org/abs/2611.21797)

## Source-stated

The authors say test quality declined sharply with project size, and that more tests accounted for much of the coverage advantage.

> agent test quality degrades sharply with project size, and much of the advantage lies in a greater number of written tests.

- [Supporting source](https://arxiv.org/abs/2611.21797)

## Source-stated

The authors characterize test translation by agents as inconsistent and unreliable despite strong aggregate results.

> agentic test translation---while seemingly strong in aggregate---remains inconsistent and unreliable.

- [Supporting source](https://arxiv.org/abs/2611.21797)

## Escape Meridian interpretation

Passing tests may not establish confidence in agent-written code when the agents also write the tests; this is an interpretation of the authors’ stated concern and findings.

> Much of this trust rests on passing tests, which are often written by the coding agents themselves.

- [Supporting source](https://arxiv.org/abs/2611.21797)

## Source and contributor context

Pasternak, Gil, Langley, AJ, Yotam, Tom, Kitanidis, Ellie · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

- [Definition reference](https://arxiv.org/abs/2211.09110)

## Frontier model

One of the most capable AI models available at a given time; the set shifts as models improve.

- [Definition reference](https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper)

## Agentic AI

AI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer.

- [Definition reference](https://developers.google.com/machine-learning/glossary#agentic)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: 8f0bd417-2c81-42d6-8266-ae737ae33948.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-18T14:48:58.699Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 91391d95b49b50a272f3cb1e250f25ea4dabf5712066cf337fa726761cbadd2c. Source text hash: 2ff3cd06859baae4317c1aaaf574ea7fb3cac08f5b79c8cf821e83fc2784f911.


### passed — 2026-11-18T14:48:58.699Z — anthropic/claude-opus-5.5

Publication: 8f0bd417-2c81-42d6-8266-ae737ae33948

attribution — pass: The reading names the four listed authors and attributes the findings to them with phrases such as 'the authors report' and 'they say.' The interpretive sentence is explicitly labeled as Escape Meridian interpretation.

> Authors:Gil Pasternak, AJ Langley, Tom Yotam, Ellie Kitanidis

summary — pass: The reading leads with the question the authors study and the trust problem they raise. It then gives the benchmark setup, the coverage result, its caveats and the translation finding, and it explains transpiling in plain words. 'Coverage,' 'frontier model' and 'reasoning effort' are kept as exact terms and should get tooltips; a coverage tooltip (how much of the code tests exercise) is suggested. Minor fidelity loss: the source says agents 'far exceed' human suites. Restoring 'far' would preserve that force. The question asks about a real gap in the abstract.

> we find that coding agents far exceed human written suites on traditional measures of coverage

date — pass: originalPublished is 2026-11-16, which matches the arXiv v1 submission date. The 2026-11-18 feed date is correctly not used.

> \[Submitted on 16 Nov 2026\]

support — pass: Each source-stated claim matches the abstract. The interpretation that agent-written passing tests complicate confidence is labeled, conditional and anchored to the authors' own framing. The limitations accurately note that the abstract gives no metrics or project sizes. No benefits or costs are invented.

> Much of this trust rests on passing tests, which are often written by the coding agents themselves.

media — pass: The selected media is the verified direct link to the arXiv abstract page. It is the most direct way to reach the source.

## Recorded references

Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter.

No public references recorded here yet.

[Reference data (JSON)](/readings/c4b349a44cc02c3dcdf1b2bdef284e13a4837eb28ff0864c793567669988c45d/references.json)

## Corrections and challenges

[Send a correction or an opposing source](https://escapemeridian.com/corrections/)

[JSON record](https://escapemeridian.com/readings/can-coding-agents-write-and-transpile-unit-tests-c4b349a4.json)
