Mehtiyev, Tural, Tadesse, Biruk, Arkoh, Lawrence, Assunção, Wesley · Nov 6, 2026
When Instruction Files Go Stale: Documentation Debt and Hidden Cost in Coding Agents
Markdown · JSON · Original source
Tural Mehtiyev, Biruk Tadesse, Lawrence Arkoh and Wesley Assunção argue that stale repository instruction files can impose documentation debt: repeated costs to coding-agent sessions that may be missed when evaluation checks only whether a task succeeds. An instruction file can, for example, direct an agent to a test folder that has moved. The authors studied three stale instructions from active open-source repositories, covering a test command, a code location and a procedure for adding a component. They compared agents working on the same 22 maintenance tasks with each instruction left stale, corrected or removed, analyzing 732 trajectories from two models. Agents followed stale guidance when it was relevant, sometimes despite correct information elsewhere. For one stale test command, Sonnet 5.5 used about two more tool calls per run than with the corrected command; the authors detected no difference in the other cases. Agents generally recovered from immediate failures, but none repaired the instruction file itself. The authors say task success alone can therefore conceal costs in agent behavior. Their results point to a distinction between completing a task and avoiding unnecessary operations along the way, within the study’s tested cases.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 6, 2026 (2026-11-06).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T18:49:39.988Z. First observed: 2026-11-09T05:00:45.204Z. Retrieved: 2026-11-10T05:16:29.153Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The study covers three stale instructions from active open-source repositories and 22 maintenance tasks; the reported findings do not establish how often the same effects occur in other settings. The extra tool-call result is reported for the stale test command and Sonnet 5.5; the authors say they could not detect a difference in the other cases.
Open question
Would studies of other stale instructions, repositories and models find the same added tool-call cost, or is it limited to the test-command case reported here?
Source-stated
The authors frame stale repository instructions as documentation debt whose costs can recur across later coding-agent sessions and escape evaluations focused only on task success.
such a stale instruction is documentation debt, and its interest is the cost it adds to every later agent session.
Source-stated
In the study, agents followed stale guidance when it was relevant, sometimes even when correct information was available elsewhere.
agents follow stale guidance when it is relevant to their tasks, sometimes even when correct information is available elsewhere.
Source-stated
For the stale test command, Claude Sonnet 5.5 used about two extra tool calls per run compared with the corrected command; the authors detected no difference in the other cases.
Claude Sonnet 5.5 needed about two extra tool calls per run, a third more than with the corrected command; in the other cases, we could not detect a difference.
Source-stated
Agents generally recovered from immediate failures, but none repaired the underlying instruction file.
Although agents generally recovered from the immediate failures, none repaired the underlying instruction file.
Source-stated
The authors studied three real-world stale instructions from active open-source repositories, 22 maintenance tasks and 732 trajectories from two models.
We study three real-world stale instructions from active open-source repositories in depth
Source and contributor context
Mehtiyev, Tural, Tadesse, Biruk, Arkoh, Lawrence, Assunção, Wesley · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Claude
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Editorial provenance
Version 1. Publication ID: 30c0ac86-132e-4d31-8feb-0445b37a9db8.
Prepared by server-excerpt-repair/stream-excerpt-repair-v1 (original draft: gpt-6-luna); grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T18:49:39.988Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: useful_perspective.
Observation: 1f7b29a020462ad80fda1e1320f6ef4695dc408352882d07715e0f4fb3fa2c65. Source text hash: e024f23fe382b9b39ed14b94dea1a14c7a0d4d2dcca81ac93dc1546f1906772a.
Revision and source-check history
Version 1 · 2026-11-10T18:49:39.988Z
When Instruction Files Go Stale: Documentation Debt and Hidden Cost in Coding Agents
Tural Mehtiyev, Biruk Tadesse, Lawrence Arkoh and Wesley Assunção argue that stale repository instruction files can impose documentation debt: repeated costs to coding-agent sessions that may be missed when evaluation checks only whether a task succeeds. An instruction file can, for example, direct an agent to a test folder that has moved. The authors studied three stale instructions from active open-source repositories, covering a test command, a code location and a procedure for adding a component. They compared agents working on the same 22 maintenance tasks with each instruction left stale, corrected or removed, analyzing 732 trajectories from two models. Agents followed stale guidance when it was relevant, sometimes despite correct information elsewhere. For one stale test command, Sonnet 5.5 used about two more tool calls per run than with the corrected command; the authors detected no difference in the other cases. Agents generally recovered from immediate failures, but none repaired the instruction file itself. The authors say task success alone can therefore conceal costs in agent behavior. Their results point to a distinction between completing a task and avoiding unnecessary operations along the way, within the study’s tested cases.
Original source
cs.AI · Article
Source details
Original source date: Nov 6, 2026 (2026-11-06).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T18:49:39.988Z. First observed: 2026-11-09T05:00:45.204Z. Retrieved: 2026-11-10T05:16:29.153Z.
Limits
The study covers three stale instructions from active open-source repositories and 22 maintenance tasks; the reported findings do not establish how often the same effects occur in other settings. The extra tool-call result is reported for the stale test command and Sonnet 5.5; the authors say they could not detect a difference in the other cases.
Open question
Would studies of other stale instructions, repositories and models find the same added tool-call cost, or is it limited to the test-command case reported here?
Source-stated
The authors frame stale repository instructions as documentation debt whose costs can recur across later coding-agent sessions and escape evaluations focused only on task success.
such a stale instruction is documentation debt, and its interest is the cost it adds to every later agent session.
Source-stated
In the study, agents followed stale guidance when it was relevant, sometimes even when correct information was available elsewhere.
agents follow stale guidance when it is relevant to their tasks, sometimes even when correct information is available elsewhere.
Source-stated
For the stale test command, Claude Sonnet 5.5 used about two extra tool calls per run compared with the corrected command; the authors detected no difference in the other cases.
Claude Sonnet 5.5 needed about two extra tool calls per run, a third more than with the corrected command; in the other cases, we could not detect a difference.
Source-stated
Agents generally recovered from immediate failures, but none repaired the underlying instruction file.
Although agents generally recovered from the immediate failures, none repaired the underlying instruction file.
Source-stated
The authors studied three real-world stale instructions from active open-source repositories, 22 maintenance tasks and 732 trajectories from two models.
We study three real-world stale instructions from active open-source repositories in depth
Source and contributor context
Mehtiyev, Tural, Tadesse, Biruk, Arkoh, Lawrence, Assunção, Wesley · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Claude
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Editorial provenance
Version 1. Publication ID: 30c0ac86-132e-4d31-8feb-0445b37a9db8.
Prepared by server-excerpt-repair/stream-excerpt-repair-v1 (original draft: gpt-6-luna); grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T18:49:39.988Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: useful_perspective.
Observation: 1f7b29a020462ad80fda1e1320f6ef4695dc408352882d07715e0f4fb3fa2c65. Source text hash: e024f23fe382b9b39ed14b94dea1a14c7a0d4d2dcca81ac93dc1546f1906772a.
uncertain · 2026-11-10T18:49:39.988Z · anthropic/claude-opus-5.5
Publication: 30c0ac86-132e-4d31-8feb-0445b37a9db8
attribution — pass: The draft correctly attributes the study and its framing to Tural Mehtiyev, Biruk Tadesse, Lawrence Arkoh and Wesley Assunção. It presents the findings as the authors' results, not as established fact.
Authors:Tural Mehtiyev, Biruk Tadesse, Lawrence Arkoh, Wesley Assunção
summary — uncertain: The draft is faithful but omits a central comparison result. Correcting the instruction eliminated operations on obsolete paths, while removing it prevented most of them. That stale/corrected/removed contrast is the core of the design. Smallest repair: add one sentence with this result. Two terms are also unexplained for general readers: 'tool calls' (actions such as running commands or opening files) and 'trajectories' (recorded agent runs). Term-library tooltips would fix this. The closing 'distinction' sentence is mild interpretation and is acceptably scoped.
Correcting the instructions eliminated operations on obsolete paths, while removing them prevented most.
date — pass: The originalPublished date of 2026-11-06 matches the v1 submission date. It is not the later feed date of 2026-11-09.
[Submitted on 6 Nov 2026]
support — pass: The source text supports every claim, limitation and the question. 'Detected no difference' fairly renders 'could not detect a difference.' The claim 5 excerpt covers only the three instructions, but the abstract also states the 22 tasks and 732 trajectories from two models. The question follows from the reported scope.
Agents fix 22 maintenance tasks under three versions of each instruction (left stale, corrected to the current path, and removed)
media — pass: The selected media is the verified direct link to the original arXiv abstract page. That page is the source itself.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Explore ClaudearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivClaude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Explore ClaudeClaude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Explore ClaudearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivClaude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Explore Claude