{"id":"5ee41bb9a0c0ecb622f115a995cdbdd6bc033b2843cbc13d2c6fb3f7fb516aee","title":"Wang and co-authors introduce FeatureArc for testing coding agents on evolving features","argument":"Yunlong Wang and co-authors introduce FeatureArc, a benchmark for testing coding agents as software features are extended through successive stages. Unlike tests of a single request on a fixed codebase, FeatureArc checks later work against earlier code and tests whether existing behavior still works.\n\nThe authors evaluated 10 coding agents on 73 manually reviewed tasks, each with 2 to 10 stages. In their evaluation, pooled success fell from 62.7% at the first stage to 28.5% when the whole feature was tested on the final code. They report that inherited defects appeared in 72.9% of failed later stages, and that agents left earlier errors unnoticed about 95% of the time in stages following a failure.\n\nThese results point to a challenge in extending software over multiple stages without human checks between them. They do not establish how agents perform in other tasks or settings. How would the results change with different repositories, agents, or human checks between stages?","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evaluation covered 10 coding agents and 73 manually reviewed tasks, so the evidence does not establish how results generalize to other agents or tasks. The agents worked in one persistent session and workspace without external test feedback or rollback; the source does not report how results compare with other workflows. The source provides no evidence here about performance in production software or the effects of human checks between stages.","speaker":{"name":"Wang, Yunlong, Wu, Alyssa, Gao, Cuiyun, Feng, Jia, Lu, Songyuanyi, Hu, Zhichao","roleAtSource":"Article author"},"source":{"title":"FeatureArc: Benchmarking Coding Agents on Evolving Features","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.37434","publishedAt":"2026-10-27","dateBasis":"Date basis not provided."},"evidence":[{"text":"Wang and co-authors introduce FeatureArc to evaluate coding agents on successive extensions of a feature, using tasks built from repository histories.","excerpt":"We introduce FeatureArc to evaluate coding agents on evolving features, together with a pipeline that constructs its tasks from repository histories.","url":"https://arxiv.org/abs/2610.37434","relationship":"Source-stated"},{"text":"The evaluation covered 10 coding agents and 73 manually reviewed tasks, each with 2 to 10 stages.","excerpt":"We evaluated 10 coding agents on 73 manually reviewed tasks of 2 to 10 stages.","url":"https://arxiv.org/abs/2610.37434","relationship":"Source-stated"},{"text":"Pooled success was lower when agents were evaluated on the whole feature at the final stage than at the first stage.","excerpt":"pooled success falls from 62.7% at the first stage to 28.5% when the whole feature is tested on the final code.","url":"https://arxiv.org/abs/2610.37434","relationship":"Source-stated"},{"text":"The authors report inherited defects as a common failure mechanism in later stages, and say agents often did not notice earlier errors after a failure.","excerpt":"inherited defects as a common failure mechanism, present in 72.9% of failed later stages. These earlier errors often remain undetected: in stages following a failure, agents leave the earlier error unnoticed about 95% of the time","url":"https://arxiv.org/abs/2610.37434","relationship":"Source-stated"},{"text":"The results indicate a challenge for autonomous feature development without human checks between stages; the evidence does not show how broadly the findings generalize.","excerpt":"Evolving a feature autonomously, without human checks between stages, therefore remains a substantial challenge for current coding agents.","url":"https://arxiv.org/abs/2610.37434","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.37434","checkedAt":"2026-11-07T03:24:06.669Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How would FeatureArc results change with different repositories, coding agents, or human checks between stages?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Wang and co-authors introduce FeatureArc for testing coding agents on evolving features","summary":"Yunlong Wang and co-authors introduce FeatureArc, a benchmark for testing coding agents as software features are extended through successive stages. Unlike tests of a single request on a fixed codebase, FeatureArc checks later work against earlier code and tests whether existing behavior still works.\n\nThe authors evaluated 10 coding agents on 73 manually reviewed tasks, each with 2 to 10 stages. In their evaluation, pooled success fell from 62.7% at the first stage to 28.5% when the whole feature was tested on the final code. They report that inherited defects appeared in 72.9% of failed later stages, and that agents left earlier errors unnoticed about 95% of the time in stages following a failure.\n\nThese results point to a challenge in extending software over multiple stages without human checks between them. They do not establish how agents perform in other tasks or settings. How would the results change with different repositories, agents, or human checks between stages?","question":"How would FeatureArc results change with different repositories, coding agents, or human checks between stages?","claims":[{"text":"Wang and co-authors introduce FeatureArc to evaluate coding agents on successive extensions of a feature, using tasks built from repository histories.","excerpt":"We introduce FeatureArc to evaluate coding agents on evolving features, together with a pipeline that constructs its tasks from repository histories.","support":"source-stated"},{"text":"The evaluation covered 10 coding agents and 73 manually reviewed tasks, each with 2 to 10 stages.","excerpt":"We evaluated 10 coding agents on 73 manually reviewed tasks of 2 to 10 stages.","support":"source-stated"},{"text":"Pooled success was lower when agents were evaluated on the whole feature at the final stage than at the first stage.","excerpt":"pooled success falls from 62.7% at the first stage to 28.5% when the whole feature is tested on the final code.","support":"source-stated"},{"text":"The authors report inherited defects as a common failure mechanism in later stages, and say agents often did not notice earlier errors after a failure.","excerpt":"inherited defects as a common failure mechanism, present in 72.9% of failed later stages. These earlier errors often remain undetected: in stages following a failure, agents leave the earlier error unnoticed about 95% of the time","support":"source-stated"},{"text":"The results indicate a challenge for autonomous feature development without human checks between stages; the evidence does not show how broadly the findings generalize.","excerpt":"Evolving a feature autonomously, without human checks between stages, therefore remains a substantial challenge for current coding agents.","support":"interpretation"}],"limitations":["The evaluation covered 10 coding agents and 73 manually reviewed tasks, so the evidence does not establish how results generalize to other agents or tasks.","The agents worked in one persistent session and workspace without external test feedback or rollback; the source does not report how results compare with other workflows.","The source provides no evidence here about performance in production software or the effects of human checks between stages."],"mediaId":"72d724b7d33092618e7f1268d33c0ce66dfda797b55115d9045963e9ac9673ee","correctionReason":null,"id":"5ee41bb9a0c0ecb622f115a995cdbdd6bc033b2843cbc13d2c6fb3f7fb516aee","publicationId":"efbf0cbe-69a9-470d-8d13-fca4e40104b7","version":1,"sourceUrl":"https://arxiv.org/abs/2610.37434","sourceTitle":"FeatureArc: Benchmarking Coding Agents on Evolving Features","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-27","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-28T07:18:33.838Z","createdAt":"2026-11-07T01:17:26.036Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"72d724b7d33092618e7f1268d33c0ce66dfda797b55115d9045963e9ac9673ee","kind":"article","url":"https://arxiv.org/abs/2610.37434","sourceUrl":"https://arxiv.org/abs/2610.37434","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-30T01:03:20.929Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.37434","checkedAt":"2026-11-07T03:24:06.669Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"72d724b7d33092618e7f1268d33c0ce66dfda797b55115d9045963e9ac9673ee","confidence":0.85,"provider":"typesafe","model":"jev-1.13.0","inputHash":"6064e3910aada305940de1a280f4e93dd5378effa8561ebb7a95c7153ef3a7bc","decidedAt":"2026-10-30T01:03:24.256Z"},"provenance":{"textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observationId":"3ae0ebb4198e29135e5b205376321d9fc9ceeb8fe2977b92ed72b8c440211f51","retrievedAt":"2026-10-30T01:03:20.454Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"9839d90f-1b75-4bb6-81f1-fa8a428fecc6"},"reviewer":{"model":"gpt-6-luna","runId":"a6aa48fc-e52a-4b36-8945-b6212801d390"}},"audit":{"status":"passed","reviewedAt":"2026-11-07T01:17:26.036Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Wang, Yunlong","url":null,"kind":"unknown"},{"name":"Wu, Alyssa","url":null,"kind":"unknown"},{"name":"Gao, Cuiyun","url":null,"kind":"unknown"},{"name":"Feng, Jia","url":null,"kind":"unknown"},{"name":"Lu, Songyuanyi","url":null,"kind":"unknown"},{"name":"Hu, Zhichao","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Wang, Yunlong","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wang, Yunlong","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.37434","locator":"head/meta[name=author|citation_author]"},"assertionId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-07T03:24:06.669Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","sourceUrl":"https://arxiv.org/abs/2610.37434","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z"}},{"actor":{"name":"Wu, Alyssa","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wu, Alyssa","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.37434","locator":"head/meta[name=author|citation_author]"},"assertionId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-07T03:24:06.669Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","sourceUrl":"https://arxiv.org/abs/2610.37434","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z"}},{"actor":{"name":"Gao, Cuiyun","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Gao, Cuiyun","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.37434","locator":"head/meta[name=author|citation_author]"},"assertionId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-07T03:24:06.669Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","sourceUrl":"https://arxiv.org/abs/2610.37434","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z"}},{"actor":{"name":"Feng, Jia","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Feng, Jia","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.37434","locator":"head/meta[name=author|citation_author]"},"assertionId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-07T03:24:06.669Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","sourceUrl":"https://arxiv.org/abs/2610.37434","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z"}},{"actor":{"name":"Lu, Songyuanyi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Lu, Songyuanyi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.37434","locator":"head/meta[name=author|citation_author]"},"assertionId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-07T03:24:06.669Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","sourceUrl":"https://arxiv.org/abs/2610.37434","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z"}},{"actor":{"name":"Hu, Zhichao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Hu, Zhichao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.37434","locator":"head/meta[name=author|citation_author]"},"assertionId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-07T03:24:06.669Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","sourceUrl":"https://arxiv.org/abs/2610.37434","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z"}}],"participationCoverage":{"total":6,"returned":6,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.37434","snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z","basis":"publisher-page-metadata","sourceLinks":45,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/5ee41bb9a0c0ecb622f115a995cdbdd6bc033b2843cbc13d2c6fb3f7fb516aee/source"}},"canonicalUrl":"https://escapemeridian.com/readings/wang-and-co-authors-introduce-featurearc-for-testing-coding-5ee41bb9","formats":{"html":"/readings/wang-and-co-authors-introduce-featurearc-for-testing-coding-5ee41bb9","markdown":"/readings/wang-and-co-authors-introduce-featurearc-for-testing-coding-5ee41bb9.md","json":"/readings/wang-and-co-authors-introduce-featurearc-for-testing-coding-5ee41bb9.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"},{"id":"time","name":"TIME","kind":"organization","context":"TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.","referenceUrls":["https://time.com/"],"path":"/entities/time","json":"/entities/time.json","markdown":"/entities/time.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"5ee41bb9a0c0ecb622f115a995cdbdd6bc033b2843cbc13d2c6fb3f7fb516aee","revision":"efbf0cbe-69a9-470d-8d13-fca4e40104b7","revisionKind":"publication-id","title":"Wang and co-authors introduce FeatureArc for testing coding agents on evolving features","url":"https://escapemeridian.com/readings/wang-and-co-authors-introduce-featurearc-for-testing-coding-5ee41bb9"},"source":{"type":"source","id":"https://arxiv.org/abs/2610.37434","revision":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","revisionKind":"sha256-source-text","title":"FeatureArc: Benchmarking Coding Agents on Evolving Features","url":"https://arxiv.org/abs/2610.37434"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"5ee41bb9a0c0ecb622f115a995cdbdd6bc033b2843cbc13d2c6fb3f7fb516aee","title":"Wang and co-authors introduce FeatureArc for testing coding agents on evolving features","argument":"Yunlong Wang and co-authors introduce FeatureArc, a benchmark for testing coding agents as software features are extended through successive stages. Unlike tests of a single request on a fixed codebase, FeatureArc checks later work against earlier code and tests whether existing behavior still works.\n\nThe authors evaluated 10 coding agents on 73 manually reviewed tasks, each with 2 to 10 stages. In their evaluation, pooled success fell from 62.7% at the first stage to 28.5% when the whole feature was tested on the final code. They report that inherited defects appeared in 72.9% of failed later stages, and that agents left earlier errors unnoticed about 95% of the time in stages following a failure.\n\nThese results point to a challenge in extending software over multiple stages without human checks between them. They do not establish how agents perform in other tasks or settings. How would the results change with different repositories, agents, or human checks between stages?","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evaluation covered 10 coding agents and 73 manually reviewed tasks, so the evidence does not establish how results generalize to other agents or tasks. The agents worked in one persistent session and workspace without external test feedback or rollback; the source does not report how results compare with other workflows. The source provides no evidence here about performance in production software or the effects of human checks between stages.","speaker":{"name":"Wang, Yunlong, Wu, Alyssa, Gao, Cuiyun, Feng, Jia, Lu, Songyuanyi, Hu, Zhichao","roleAtSource":"Article author"},"source":{"title":"FeatureArc: Benchmarking Coding Agents on Evolving Features","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.37434","publishedAt":"2026-10-27","dateBasis":"Date basis not provided."},"evidence":[{"text":"Wang and co-authors introduce FeatureArc to evaluate coding agents on successive extensions of a feature, using tasks built from repository histories.","excerpt":"We introduce FeatureArc to evaluate coding agents on evolving features, together with a pipeline that constructs its tasks from repository histories.","url":"https://arxiv.org/abs/2610.37434","relationship":"Source-stated"},{"text":"The evaluation covered 10 coding agents and 73 manually reviewed tasks, each with 2 to 10 stages.","excerpt":"We evaluated 10 coding agents on 73 manually reviewed tasks of 2 to 10 stages.","url":"https://arxiv.org/abs/2610.37434","relationship":"Source-stated"},{"text":"Pooled success was lower when agents were evaluated on the whole feature at the final stage than at the first stage.","excerpt":"pooled success falls from 62.7% at the first stage to 28.5% when the whole feature is tested on the final code.","url":"https://arxiv.org/abs/2610.37434","relationship":"Source-stated"},{"text":"The authors report inherited defects as a common failure mechanism in later stages, and say agents often did not notice earlier errors after a failure.","excerpt":"inherited defects as a common failure mechanism, present in 72.9% of failed later stages. These earlier errors often remain undetected: in stages following a failure, agents leave the earlier error unnoticed about 95% of the time","url":"https://arxiv.org/abs/2610.37434","relationship":"Source-stated"},{"text":"The results indicate a challenge for autonomous feature development without human checks between stages; the evidence does not show how broadly the findings generalize.","excerpt":"Evolving a feature autonomously, without human checks between stages, therefore remains a substantial challenge for current coding agents.","url":"https://arxiv.org/abs/2610.37434","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.37434","checkedAt":"2026-11-07T03:24:06.669Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How would FeatureArc results change with different repositories, coding agents, or human checks between stages?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Wang and co-authors introduce FeatureArc for testing coding agents on evolving features","summary":"Yunlong Wang and co-authors introduce FeatureArc, a benchmark for testing coding agents as software features are extended through successive stages. Unlike tests of a single request on a fixed codebase, FeatureArc checks later work against earlier code and tests whether existing behavior still works.\n\nThe authors evaluated 10 coding agents on 73 manually reviewed tasks, each with 2 to 10 stages. In their evaluation, pooled success fell from 62.7% at the first stage to 28.5% when the whole feature was tested on the final code. They report that inherited defects appeared in 72.9% of failed later stages, and that agents left earlier errors unnoticed about 95% of the time in stages following a failure.\n\nThese results point to a challenge in extending software over multiple stages without human checks between them. They do not establish how agents perform in other tasks or settings. How would the results change with different repositories, agents, or human checks between stages?","question":"How would FeatureArc results change with different repositories, coding agents, or human checks between stages?","claims":[{"text":"Wang and co-authors introduce FeatureArc to evaluate coding agents on successive extensions of a feature, using tasks built from repository histories.","excerpt":"We introduce FeatureArc to evaluate coding agents on evolving features, together with a pipeline that constructs its tasks from repository histories.","support":"source-stated"},{"text":"The evaluation covered 10 coding agents and 73 manually reviewed tasks, each with 2 to 10 stages.","excerpt":"We evaluated 10 coding agents on 73 manually reviewed tasks of 2 to 10 stages.","support":"source-stated"},{"text":"Pooled success was lower when agents were evaluated on the whole feature at the final stage than at the first stage.","excerpt":"pooled success falls from 62.7% at the first stage to 28.5% when the whole feature is tested on the final code.","support":"source-stated"},{"text":"The authors report inherited defects as a common failure mechanism in later stages, and say agents often did not notice earlier errors after a failure.","excerpt":"inherited defects as a common failure mechanism, present in 72.9% of failed later stages. These earlier errors often remain undetected: in stages following a failure, agents leave the earlier error unnoticed about 95% of the time","support":"source-stated"},{"text":"The results indicate a challenge for autonomous feature development without human checks between stages; the evidence does not show how broadly the findings generalize.","excerpt":"Evolving a feature autonomously, without human checks between stages, therefore remains a substantial challenge for current coding agents.","support":"interpretation"}],"limitations":["The evaluation covered 10 coding agents and 73 manually reviewed tasks, so the evidence does not establish how results generalize to other agents or tasks.","The agents worked in one persistent session and workspace without external test feedback or rollback; the source does not report how results compare with other workflows.","The source provides no evidence here about performance in production software or the effects of human checks between stages."],"mediaId":"72d724b7d33092618e7f1268d33c0ce66dfda797b55115d9045963e9ac9673ee","correctionReason":null,"id":"5ee41bb9a0c0ecb622f115a995cdbdd6bc033b2843cbc13d2c6fb3f7fb516aee","publicationId":"efbf0cbe-69a9-470d-8d13-fca4e40104b7","version":1,"sourceUrl":"https://arxiv.org/abs/2610.37434","sourceTitle":"FeatureArc: Benchmarking Coding Agents on Evolving Features","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-27","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-28T07:18:33.838Z","createdAt":"2026-11-07T01:17:26.036Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"72d724b7d33092618e7f1268d33c0ce66dfda797b55115d9045963e9ac9673ee","kind":"article","url":"https://arxiv.org/abs/2610.37434","sourceUrl":"https://arxiv.org/abs/2610.37434","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-30T01:03:20.929Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.37434","checkedAt":"2026-11-07T03:24:06.669Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"72d724b7d33092618e7f1268d33c0ce66dfda797b55115d9045963e9ac9673ee","confidence":0.85,"provider":"typesafe","model":"jev-1.13.0","inputHash":"6064e3910aada305940de1a280f4e93dd5378effa8561ebb7a95c7153ef3a7bc","decidedAt":"2026-10-30T01:03:24.256Z"},"provenance":{"textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observationId":"3ae0ebb4198e29135e5b205376321d9fc9ceeb8fe2977b92ed72b8c440211f51","retrievedAt":"2026-10-30T01:03:20.454Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"9839d90f-1b75-4bb6-81f1-fa8a428fecc6"},"reviewer":{"model":"gpt-6-luna","runId":"a6aa48fc-e52a-4b36-8945-b6212801d390"}},"audit":{"status":"passed","reviewedAt":"2026-11-07T01:17:26.036Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Wang, Yunlong","url":null,"kind":"unknown"},{"name":"Wu, Alyssa","url":null,"kind":"unknown"},{"name":"Gao, Cuiyun","url":null,"kind":"unknown"},{"name":"Feng, Jia","url":null,"kind":"unknown"},{"name":"Lu, Songyuanyi","url":null,"kind":"unknown"},{"name":"Hu, Zhichao","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Wang, Yunlong","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wang, Yunlong","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.37434","locator":"head/meta[name=author|citation_author]"},"assertionId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-07T03:24:06.669Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","sourceUrl":"https://arxiv.org/abs/2610.37434","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z"}},{"actor":{"name":"Wu, Alyssa","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wu, Alyssa","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.37434","locator":"head/meta[name=author|citation_author]"},"assertionId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-07T03:24:06.669Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","sourceUrl":"https://arxiv.org/abs/2610.37434","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z"}},{"actor":{"name":"Gao, Cuiyun","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Gao, Cuiyun","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.37434","locator":"head/meta[name=author|citation_author]"},"assertionId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-07T03:24:06.669Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","sourceUrl":"https://arxiv.org/abs/2610.37434","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z"}},{"actor":{"name":"Feng, Jia","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Feng, Jia","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.37434","locator":"head/meta[name=author|citation_author]"},"assertionId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-07T03:24:06.669Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","sourceUrl":"https://arxiv.org/abs/2610.37434","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z"}},{"actor":{"name":"Lu, Songyuanyi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Lu, Songyuanyi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.37434","locator":"head/meta[name=author|citation_author]"},"assertionId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-07T03:24:06.669Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","sourceUrl":"https://arxiv.org/abs/2610.37434","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z"}},{"actor":{"name":"Hu, Zhichao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Hu, Zhichao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.37434","locator":"head/meta[name=author|citation_author]"},"assertionId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-07T03:24:06.669Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","sourceUrl":"https://arxiv.org/abs/2610.37434","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z"}}],"participationCoverage":{"total":6,"returned":6,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.37434","snapshotId":"6e62ab137feadd654e59a119d59855e330f82d5c44c0d475959e189dd7a4ac0e","contentSha256":"3ec0cc11cd070ba43cb4449360cbed6742686f27620c18c5b763d760898e421b","textHash":"02534a21f4a3812f447adf0b08275d09f81f53b5f52473039cfa7e6edcecbd14","observedAt":"2026-11-07T03:24:06.669Z","basis":"publisher-page-metadata","sourceLinks":45,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/5ee41bb9a0c0ecb622f115a995cdbdd6bc033b2843cbc13d2c6fb3f7fb516aee/source"}}}],"audits":[{"publicationId":"efbf0cbe-69a9-470d-8d13-fca4e40104b7","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading correctly credits Yunlong Wang and co-authors with introducing FeatureArc and keeps the results attributed to their evaluation (\"The authors evaluated\", \"They report\").","excerpt":"Authors:Yunlong Wang, Alyssa Wu, Cuiyun Gao, Jia Feng, Songyuanyi Lu, Zhichao Hu"},{"dimension":"summary","status":"pass","explanation":"The summary leads with what FeatureArc is, contrasts it in plain language with single-request benchmarks, and gives the key numbers and the failure mechanism of inherited defects. It ends with the authors' own conclusion and a non-leading question. The persistent-session, no-feedback, no-rollback condition is kept in the limitations. The Tencent/Hunyuan use is omitted, but this does not distort the meaning. Terms such as pooled success can be handled with tooltips.","excerpt":"Each agent works in one persistent session and workspace without external test feedback or rollback."},{"dimension":"date","status":"pass","explanation":"The originalPublished date of 2026-10-27 matches the arXiv v1 submission date. It is not the later feed date of 2026-10-29.","excerpt":"[Submitted on 27 Oct 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim matches its verbatim excerpt. The figures 62.7%, 28.5%, 72.9% and about 95% are reproduced accurately. The interpretation claim stays close to the authors' stated conclusion and adds an appropriate caution about how far the findings generalize. The limitations accurately note the narrow scope and that the evaluation did not include human checks between stages.","excerpt":"pooled success falls from 62.7% at the first stage to 28.5% when the whole feature is tested on the final code."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the original arXiv abstract page, which matches the source URL. It is the most direct way to access the paper.","excerpt":null}],"createdAt":"2026-11-07T01:17:26.036Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}