# Wang and co-authors introduce FeatureArc for testing coding agents on evolving features [Canonical reading](https://escapemeridian.com/readings/wang-and-co-authors-introduce-featurearc-for-testing-coding-5ee41bb9) Yunlong Wang and co-authors introduce FeatureArc, a benchmark for testing coding agents as software features are extended through successive stages. Unlike tests of a single request on a fixed codebase, FeatureArc checks later work against earlier code and tests whether existing behavior still works. The authors evaluated 10 coding agents on 73 manually reviewed tasks, each with 2 to 10 stages. In their evaluation, pooled success fell from 62.7% at the first stage to 28.5% when the whole feature was tested on the final code. They report that inherited defects appeared in 72.9% of failed later stages, and that agents left earlier errors unnoticed about 95% of the time in stages following a failure. These results point to a challenge in extending software over multiple stages without human checks between them. They do not establish how agents perform in other tasks or settings. How would the results change with different repositories, agents, or human checks between stages? Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Oct 27, 2026 (2026-10-27). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-07T01:17:26.036Z. First observed: 2026-10-28T07:18:33.838Z. Retrieved: 2026-10-30T01:03:20.454Z. - [FeatureArc: Benchmarking Coding Agents on Evolving Features](https://arxiv.org/abs/2610.37434) ## Limits The evaluation covered 10 coding agents and 73 manually reviewed tasks, so the evidence does not establish how results generalize to other agents or tasks. The agents worked in one persistent session and workspace without external test feedback or rollback; the source does not report how results compare with other workflows. The source provides no evidence here about performance in production software or the effects of human checks between stages. ## Open question How would FeatureArc results change with different repositories, coding agents, or human checks between stages? ## Source-stated Wang and co-authors introduce FeatureArc to evaluate coding agents on successive extensions of a feature, using tasks built from repository histories. > We introduce FeatureArc to evaluate coding agents on evolving features, together with a pipeline that constructs its tasks from repository histories. - [Supporting source](https://arxiv.org/abs/2610.37434) ## Source-stated The evaluation covered 10 coding agents and 73 manually reviewed tasks, each with 2 to 10 stages. > We evaluated 10 coding agents on 73 manually reviewed tasks of 2 to 10 stages. - [Supporting source](https://arxiv.org/abs/2610.37434) ## Source-stated Pooled success was lower when agents were evaluated on the whole feature at the final stage than at the first stage. > pooled success falls from 62.7% at the first stage to 28.5% when the whole feature is tested on the final code. - [Supporting source](https://arxiv.org/abs/2610.37434) ## Source-stated The authors report inherited defects as a common failure mechanism in later stages, and say agents often did not notice earlier errors after a failure. > inherited defects as a common failure mechanism, present in 72.9% of failed later stages. These earlier errors often remain undetected: in stages following a failure, agents leave the earlier error unnoticed about 95% of the time - [Supporting source](https://arxiv.org/abs/2610.37434) ## Escape Meridian interpretation The results indicate a challenge for autonomous feature development without human checks between stages; the evidence does not show how broadly the findings generalize. > Evolving a feature autonomously, without human checks between stages, therefore remains a substantial challenge for current coding agents. - [Supporting source](https://arxiv.org/abs/2610.37434) ## Source and contributor context Wang, Yunlong, Wu, Alyssa, Gao, Cuiyun, Feng, Jia, Lu, Songyuanyi, Hu, Zhichao · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## TIME TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services. - [Explore this context](/entities/time) - [Reference](https://time.com/) ## Editorial provenance Version 1. Publication ID: efbf0cbe-69a9-470d-8d13-fca4e40104b7. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-07T01:17:26.036Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 3ae0ebb4198e29135e5b205376321d9fc9ceeb8fe2977b92ed72b8c440211f51. Source text hash: 02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14. ## Revision and source-check history ### Version 1 — 2026-11-07T01:17:26.036Z Yunlong Wang and co-authors introduce FeatureArc, a benchmark for testing coding agents as software features are extended through successive stages. Unlike tests of a single request on a fixed codebase, FeatureArc checks later work against earlier code and tests whether existing behavior still works. The authors evaluated 10 coding agents on 73 manually reviewed tasks, each with 2 to 10 stages. In their evaluation, pooled success fell from 62.7% at the first stage to 28.5% when the whole feature was tested on the final code. They report that inherited defects appeared in 72.9% of failed later stages, and that agents left earlier errors unnoticed about 95% of the time in stages following a failure. These results point to a challenge in extending software over multiple stages without human checks between them. They do not establish how agents perform in other tasks or settings. How would the results change with different repositories, agents, or human checks between stages? ## Original source arXiv cs.AI · Article Original source date: Oct 27, 2026 (2026-10-27). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-07T01:17:26.036Z. First observed: 2026-10-28T07:18:33.838Z. Retrieved: 2026-10-30T01:03:20.454Z. - [FeatureArc: Benchmarking Coding Agents on Evolving Features](https://arxiv.org/abs/2610.37434) ## Limits The evaluation covered 10 coding agents and 73 manually reviewed tasks, so the evidence does not establish how results generalize to other agents or tasks. The agents worked in one persistent session and workspace without external test feedback or rollback; the source does not report how results compare with other workflows. The source provides no evidence here about performance in production software or the effects of human checks between stages. ## Open question How would FeatureArc results change with different repositories, coding agents, or human checks between stages? ## Source-stated Wang and co-authors introduce FeatureArc to evaluate coding agents on successive extensions of a feature, using tasks built from repository histories. > We introduce FeatureArc to evaluate coding agents on evolving features, together with a pipeline that constructs its tasks from repository histories. - [Supporting source](https://arxiv.org/abs/2610.37434) ## Source-stated The evaluation covered 10 coding agents and 73 manually reviewed tasks, each with 2 to 10 stages. > We evaluated 10 coding agents on 73 manually reviewed tasks of 2 to 10 stages. - [Supporting source](https://arxiv.org/abs/2610.37434) ## Source-stated Pooled success was lower when agents were evaluated on the whole feature at the final stage than at the first stage. > pooled success falls from 62.7% at the first stage to 28.5% when the whole feature is tested on the final code. - [Supporting source](https://arxiv.org/abs/2610.37434) ## Source-stated The authors report inherited defects as a common failure mechanism in later stages, and say agents often did not notice earlier errors after a failure. > inherited defects as a common failure mechanism, present in 72.9% of failed later stages. These earlier errors often remain undetected: in stages following a failure, agents leave the earlier error unnoticed about 95% of the time - [Supporting source](https://arxiv.org/abs/2610.37434) ## Escape Meridian interpretation The results indicate a challenge for autonomous feature development without human checks between stages; the evidence does not show how broadly the findings generalize. > Evolving a feature autonomously, without human checks between stages, therefore remains a substantial challenge for current coding agents. - [Supporting source](https://arxiv.org/abs/2610.37434) ## Source and contributor context Wang, Yunlong, Wu, Alyssa, Gao, Cuiyun, Feng, Jia, Lu, Songyuanyi, Hu, Zhichao · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## TIME TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services. - [Explore this context](/entities/time) - [Reference](https://time.com/) ## Editorial provenance Version 1. Publication ID: efbf0cbe-69a9-470d-8d13-fca4e40104b7. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-07T01:17:26.036Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 3ae0ebb4198e29135e5b205376321d9fc9ceeb8fe2977b92ed72b8c440211f51. Source text hash: 02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14. ### passed — 2026-11-07T01:17:26.036Z — anthropic/claude-opus-5.5 Publication: efbf0cbe-69a9-470d-8d13-fca4e40104b7 attribution — pass: The reading correctly credits Yunlong Wang and co-authors with introducing FeatureArc and keeps the results attributed to their evaluation ("The authors evaluated", "They report"). > Authors:Yunlong Wang, Alyssa Wu, Cuiyun Gao, Jia Feng, Songyuanyi Lu, Zhichao Hu summary — pass: The summary leads with what FeatureArc is, contrasts it in plain language with single-request benchmarks, and gives the key numbers and the failure mechanism of inherited defects. It ends with the authors' own conclusion and a non-leading question. The persistent-session, no-feedback, no-rollback condition is kept in the limitations. The Tencent/Hunyuan use is omitted, but this does not distort the meaning. Terms such as pooled success can be handled with tooltips. > Each agent works in one persistent session and workspace without external test feedback or rollback. date — pass: The originalPublished date of 2026-10-27 matches the arXiv v1 submission date. It is not the later feed date of 2026-10-29. > \[Submitted on 27 Oct 2026\] support — pass: Each claim matches its verbatim excerpt. The figures 62.7%, 28.5%, 72.9% and about 95% are reproduced accurately. The interpretation claim stays close to the authors' stated conclusion and adds an appropriate caution about how far the findings generalize. The limitations accurately note the narrow scope and that the evaluation did not include human checks between stages. > pooled success falls from 62.7% at the first stage to 28.5% when the whole feature is tested on the final code. media — pass: The selected media is the verified direct link to the original arXiv abstract page, which matches the source URL. It is the most direct way to access the paper. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/5ee41bb9a0c0ecb622f115a995cdbdd6bc033b2843cbc13d2c6fb3f7fb516aee/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/wang-and-co-authors-introduce-featurearc-for-testing-coding-5ee41bb9.json)