Pasternak, Gil, Langley, AJ, Yotam, Tom, Kitanidis, Ellie · Nov 16, 2026
Can Coding Agents Write and Transpile Unit Tests
Markdown · JSON · Original source
Authors Gil Pasternak, AJ Langley, Tom Yotam and Ellie Kitanidis examine whether coding agents can write tests and translate them between programming languages. They focus on a source of trust in coding assistants: passing tests, which agents often write themselves. In a of three coding agents, each using a with high reasoning effort, the authors report that agent-written tests exceeded human-written suites on traditional coverage measures. They also report that this advantage was uneven: test quality fell sharply as project size grew, and much of the coverage advantage came from writing more tests. test translation looked strong in aggregate but remained inconsistent and unreliable, they say. Escape Meridian interpretation: these findings complicate reliance on passing tests as a sign of confidence when agents write the tests being used to judge their work. The abstract does not provide project sizes, individual results or detailed metrics, limiting how fully readers can assess the reported effects.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 16, 2026 (2026-11-16).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-18T14:48:58.699Z. First observed: 2026-11-18T05:13:48.565Z. Retrieved: 2026-11-18T14:48:03.374Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The supplied evidence is the paper’s abstract and does not give project sizes, individual results or detailed metrics for assessing the reported effects. The abstract does not specify how the authors measured test quality or test translation.
Open question
How did the authors measure test quality and test translation, and how did those results vary across project sizes?
Source-stated
The authors benchmarked three coding agents, each using a frontier model with high reasoning effort.
We benchmark three popular coding agents, each evaluated with a native frontier model and high reasoning effort.
Source-stated
The authors report that agent-written test suites exceeded human-written suites on traditional coverage measures.
coding agents far exceed human written suites on traditional measures of coverage.
Source-stated
The authors say test quality declined sharply with project size, and that more tests accounted for much of the coverage advantage.
agent test quality degrades sharply with project size, and much of the advantage lies in a greater number of written tests.
Source-stated
The authors characterize test translation by agents as inconsistent and unreliable despite strong aggregate results.
agentic test translation---while seemingly strong in aggregate---remains inconsistent and unreliable.
Escape Meridian interpretation
Passing tests may not establish confidence in agent-written code when the agents also write the tests; this is an interpretation of the authors’ stated concern and findings.
Much of this trust rests on passing tests, which are often written by the coding agents themselves.
Source and contributor context
Pasternak, Gil, Langley, AJ, Yotam, Tom, Kitanidis, Ellie · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Frontier model
One of the most capable AI models available at a given time; the set shifts as models improve.
Agentic AI
AI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 8f0bd417-2c81-42d6-8266-ae737ae33948.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-18T14:48:58.699Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 91391d95b49b50a272f3cb1e250f25ea4dabf5712066cf337fa726761cbadd2c. Source text hash: 2ff3cd06859baae4317c1aaaf574ea7fb3cac08f5b79c8cf821e83fc2784f911.
Revision and source-check history
Version 1 · 2026-11-18T14:48:58.699Z
Can Coding Agents Write and Transpile Unit Tests
Authors Gil Pasternak, AJ Langley, Tom Yotam and Ellie Kitanidis examine whether coding agents can write tests and translate them between programming languages. They focus on a source of trust in coding assistants: passing tests, which agents often write themselves. In a of three coding agents, each using a with high reasoning effort, the authors report that agent-written tests exceeded human-written suites on traditional coverage measures. They also report that this advantage was uneven: test quality fell sharply as project size grew, and much of the coverage advantage came from writing more tests. test translation looked strong in aggregate but remained inconsistent and unreliable, they say. Escape Meridian interpretation: these findings complicate reliance on passing tests as a sign of confidence when agents write the tests being used to judge their work. The abstract does not provide project sizes, individual results or detailed metrics, limiting how fully readers can assess the reported effects.
Original source
cs.AI · Article
Source details
Original source date: Nov 16, 2026 (2026-11-16).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-18T14:48:58.699Z. First observed: 2026-11-18T05:13:48.565Z. Retrieved: 2026-11-18T14:48:03.374Z.
Limits
The supplied evidence is the paper’s abstract and does not give project sizes, individual results or detailed metrics for assessing the reported effects. The abstract does not specify how the authors measured test quality or test translation.
Open question
How did the authors measure test quality and test translation, and how did those results vary across project sizes?
Source-stated
The authors benchmarked three coding agents, each using a frontier model with high reasoning effort.
We benchmark three popular coding agents, each evaluated with a native frontier model and high reasoning effort.
Source-stated
The authors report that agent-written test suites exceeded human-written suites on traditional coverage measures.
coding agents far exceed human written suites on traditional measures of coverage.
Source-stated
The authors say test quality declined sharply with project size, and that more tests accounted for much of the coverage advantage.
agent test quality degrades sharply with project size, and much of the advantage lies in a greater number of written tests.
Source-stated
The authors characterize test translation by agents as inconsistent and unreliable despite strong aggregate results.
agentic test translation---while seemingly strong in aggregate---remains inconsistent and unreliable.
Escape Meridian interpretation
Passing tests may not establish confidence in agent-written code when the agents also write the tests; this is an interpretation of the authors’ stated concern and findings.
Much of this trust rests on passing tests, which are often written by the coding agents themselves.
Source and contributor context
Pasternak, Gil, Langley, AJ, Yotam, Tom, Kitanidis, Ellie · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Frontier model
One of the most capable AI models available at a given time; the set shifts as models improve.
Agentic AI
AI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 8f0bd417-2c81-42d6-8266-ae737ae33948.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-18T14:48:58.699Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 91391d95b49b50a272f3cb1e250f25ea4dabf5712066cf337fa726761cbadd2c. Source text hash: 2ff3cd06859baae4317c1aaaf574ea7fb3cac08f5b79c8cf821e83fc2784f911.
passed · 2026-11-18T14:48:58.699Z · anthropic/claude-opus-5.5
Publication: 8f0bd417-2c81-42d6-8266-ae737ae33948
attribution — pass: The reading names the four listed authors and attributes the findings to them with phrases such as 'the authors report' and 'they say.' The interpretive sentence is explicitly labeled as Escape Meridian interpretation.
Authors:Gil Pasternak, AJ Langley, Tom Yotam, Ellie Kitanidis
summary — pass: The reading leads with the question the authors study and the trust problem they raise. It then gives the benchmark setup, the coverage result, its caveats and the translation finding, and it explains transpiling in plain words. 'Coverage,' 'frontier model' and 'reasoning effort' are kept as exact terms and should get tooltips; a coverage tooltip (how much of the code tests exercise) is suggested. Minor fidelity loss: the source says agents 'far exceed' human suites. Restoring 'far' would preserve that force. The question asks about a real gap in the abstract.
we find that coding agents far exceed human written suites on traditional measures of coverage
date — pass: originalPublished is 2026-11-16, which matches the arXiv v1 submission date. The 2026-11-18 feed date is correctly not used.
[Submitted on 16 Nov 2026]
support — pass: Each source-stated claim matches the abstract. The interpretation that agent-written passing tests complicate confidence is labeled, conditional and anchored to the authors' own framing. The limitations accurately note that the abstract gives no metrics or project sizes. No benefits or costs are invented.
Much of this trust rests on passing tests, which are often written by the coding agents themselves.
media — pass: The selected media is the verified direct link to the arXiv abstract page. It is the most direct way to reach the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceOne of the most capable AI models available at a given time; the set shifts as models improve.
Definition referenceAI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceOne of the most capable AI models available at a given time; the set shifts as models improve.
Definition referenceAI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv