Wang, Yunlong, Wu, Alyssa, Gao, Cuiyun, Feng, Jia, Lu, Songyuanyi, Hu, Zhichao · Oct 27, 2026
Wang and co-authors introduce FeatureArc for testing coding agents on evolving features
Markdown · JSON · Original source
Yunlong Wang and co-authors introduce FeatureArc, a for testing coding agents as software features are extended through successive stages. Unlike tests of a single request on a fixed codebase, FeatureArc checks later work against earlier code and tests whether existing behavior still works. The authors evaluated 10 coding agents on 73 manually reviewed tasks, each with 2 to 10 stages. In their evaluation, pooled success fell from 62.7% at the first stage to 28.5% when the whole feature was tested on the final code. They report that inherited defects appeared in 72.9% of failed later stages, and that agents left earlier errors unnoticed about 95% of the in stages following a failure. These results point to a challenge in extending software over multiple stages without human checks between them. They do not establish how agents perform in other tasks or settings. How would the results change with different repositories, agents, or human checks between stages?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 27, 2026 (2026-10-27).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-07T01:17:26.036Z. First observed: 2026-10-28T07:18:33.838Z. Retrieved: 2026-10-30T01:03:20.454Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The evaluation covered 10 coding agents and 73 manually reviewed tasks, so the evidence does not establish how results generalize to other agents or tasks. The agents worked in one persistent session and workspace without external test feedback or rollback; the source does not report how results compare with other workflows. The source provides no evidence here about performance in production software or the effects of human checks between stages.
Open question
How would FeatureArc results change with different repositories, coding agents, or human checks between stages?
Source-stated
Wang and co-authors introduce FeatureArc to evaluate coding agents on successive extensions of a feature, using tasks built from repository histories.
We introduce FeatureArc to evaluate coding agents on evolving features, together with a pipeline that constructs its tasks from repository histories.
Source-stated
The evaluation covered 10 coding agents and 73 manually reviewed tasks, each with 2 to 10 stages.
We evaluated 10 coding agents on 73 manually reviewed tasks of 2 to 10 stages.
Source-stated
Pooled success was lower when agents were evaluated on the whole feature at the final stage than at the first stage.
pooled success falls from 62.7% at the first stage to 28.5% when the whole feature is tested on the final code.
Source-stated
The authors report inherited defects as a common failure mechanism in later stages, and say agents often did not notice earlier errors after a failure.
inherited defects as a common failure mechanism, present in 72.9% of failed later stages. These earlier errors often remain undetected: in stages following a failure, agents leave the earlier error unnoticed about 95% of the time
Escape Meridian interpretation
The results indicate a challenge for autonomous feature development without human checks between stages; the evidence does not show how broadly the findings generalize.
Evolving a feature autonomously, without human checks between stages, therefore remains a substantial challenge for current coding agents.
Source and contributor context
Wang, Yunlong, Wu, Alyssa, Gao, Cuiyun, Feng, Jia, Lu, Songyuanyi, Hu, Zhichao · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: efbf0cbe-69a9-470d-8d13-fca4e40104b7.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-07T01:17:26.036Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 3ae0ebb4198e29135e5b205376321d9fc9ceeb8fe2977b92ed72b8c440211f51. Source text hash: 02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14.
Revision and source-check history
Version 1 · 2026-11-07T01:17:26.036Z
Wang and co-authors introduce FeatureArc for testing coding agents on evolving features
Yunlong Wang and co-authors introduce FeatureArc, a for testing coding agents as software features are extended through successive stages. Unlike tests of a single request on a fixed codebase, FeatureArc checks later work against earlier code and tests whether existing behavior still works. The authors evaluated 10 coding agents on 73 manually reviewed tasks, each with 2 to 10 stages. In their evaluation, pooled success fell from 62.7% at the first stage to 28.5% when the whole feature was tested on the final code. They report that inherited defects appeared in 72.9% of failed later stages, and that agents left earlier errors unnoticed about 95% of the in stages following a failure. These results point to a challenge in extending software over multiple stages without human checks between them. They do not establish how agents perform in other tasks or settings. How would the results change with different repositories, agents, or human checks between stages?
Original source
cs.AI · Article
Source details
Original source date: Oct 27, 2026 (2026-10-27).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-07T01:17:26.036Z. First observed: 2026-10-28T07:18:33.838Z. Retrieved: 2026-10-30T01:03:20.454Z.
Limits
The evaluation covered 10 coding agents and 73 manually reviewed tasks, so the evidence does not establish how results generalize to other agents or tasks. The agents worked in one persistent session and workspace without external test feedback or rollback; the source does not report how results compare with other workflows. The source provides no evidence here about performance in production software or the effects of human checks between stages.
Open question
How would FeatureArc results change with different repositories, coding agents, or human checks between stages?
Source-stated
Wang and co-authors introduce FeatureArc to evaluate coding agents on successive extensions of a feature, using tasks built from repository histories.
We introduce FeatureArc to evaluate coding agents on evolving features, together with a pipeline that constructs its tasks from repository histories.
Source-stated
The evaluation covered 10 coding agents and 73 manually reviewed tasks, each with 2 to 10 stages.
We evaluated 10 coding agents on 73 manually reviewed tasks of 2 to 10 stages.
Source-stated
Pooled success was lower when agents were evaluated on the whole feature at the final stage than at the first stage.
pooled success falls from 62.7% at the first stage to 28.5% when the whole feature is tested on the final code.
Source-stated
The authors report inherited defects as a common failure mechanism in later stages, and say agents often did not notice earlier errors after a failure.
inherited defects as a common failure mechanism, present in 72.9% of failed later stages. These earlier errors often remain undetected: in stages following a failure, agents leave the earlier error unnoticed about 95% of the time
Escape Meridian interpretation
The results indicate a challenge for autonomous feature development without human checks between stages; the evidence does not show how broadly the findings generalize.
Evolving a feature autonomously, without human checks between stages, therefore remains a substantial challenge for current coding agents.
Source and contributor context
Wang, Yunlong, Wu, Alyssa, Gao, Cuiyun, Feng, Jia, Lu, Songyuanyi, Hu, Zhichao · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: efbf0cbe-69a9-470d-8d13-fca4e40104b7.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-07T01:17:26.036Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 3ae0ebb4198e29135e5b205376321d9fc9ceeb8fe2977b92ed72b8c440211f51. Source text hash: 02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14.
passed · 2026-11-07T01:17:26.036Z · anthropic/claude-opus-5.5
Publication: efbf0cbe-69a9-470d-8d13-fca4e40104b7
attribution — pass: The reading correctly credits Yunlong Wang and co-authors with introducing FeatureArc and keeps the results attributed to their evaluation ("The authors evaluated", "They report").
Authors:Yunlong Wang, Alyssa Wu, Cuiyun Gao, Jia Feng, Songyuanyi Lu, Zhichao Hu
summary — pass: The summary leads with what FeatureArc is, contrasts it in plain language with single-request benchmarks, and gives the key numbers and the failure mechanism of inherited defects. It ends with the authors' own conclusion and a non-leading question. The persistent-session, no-feedback, no-rollback condition is kept in the limitations. The Tencent/Hunyuan use is omitted, but this does not distort the meaning. Terms such as pooled success can be handled with tooltips.
Each agent works in one persistent session and workspace without external test feedback or rollback.
date — pass: The originalPublished date of 2026-10-27 matches the arXiv v1 submission date. It is not the later feed date of 2026-10-29.
[Submitted on 27 Oct 2026]
support — pass: Each claim matches its verbatim excerpt. The figures 62.7%, 28.5%, 72.9% and about 95% are reproduced accurately. The interpretation claim stays close to the authors' stated conclusion and adds an appropriate caution about how far the findings generalize. The limitations accurately note the narrow scope and that the evaluation did not include human checks between stages.
pooled success falls from 62.7% at the first stage to 28.5% when the whole feature is tested on the final code.
media — pass: The selected media is the verified direct link to the original arXiv abstract page, which matches the source URL. It is the most direct way to access the paper.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv