Poikarayil, Oleeviya Babu, Babu, Tony Basil, Nahhas, Abdulrahman, Mehta, Het Darshan, Baktash, Jawid Ahmad, De Luca, Ernesto William, Turowski, Klaus · Oct 26, 2026
When Retrieval Is Not Enough: Obsolete Facts in Agent Memory
Markdown · JSON · Original source
The authors report that retrieving an updated fact may not be enough when an ’s memory also contains an obsolete version. In the single-hop FactConsolidation task of MemoryAgentBench, they tested Qwen2.5-7B-Instruct using BM25, a method for retrieving text by matching terms in a query. Accuracy fell from 45% to 32% when the retrieved evidence budget increased from 64 to 512 tokens, despite a gold-assisted audit finding the newest statement in every retrieved set. The authors report that replacing an unrelated statement with an obsolete competing statement reduced accuracy from 100% to 44.6% for Qwen2.5 and from 100% to 87.8% for Qwen3-14B. Their 32K-history replication also favored BM25-64 over BM25-512. These results suggest that obsolete information can interfere with using an updated fact, rather than simply adding irrelevant context. The evidence comes from a specific task and model setup, so it does not establish how often this problem occurs in deployed agents. Would the same pattern appear in other memory tasks or with different ways of retrieving and presenting updated facts?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 26, 2026 (2026-10-26).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-05T15:02:30.502Z. First observed: 2026-10-28T08:18:37.332Z. Retrieved: 2026-10-30T03:01:49.624Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The reported experiments concern the single-hop FactConsolidation task of MemoryAgentBench and specified model setups; the abstract does not establish whether the results generalize to other tasks or deployed agents. The audit that found the newest statement in each retrieved set was gold-assisted, so it does not show that an agent could identify the newest statement on its own.
Open question
Would the same pattern appear in other memory tasks or with different ways of retrieving and presenting updated facts?
Source-stated
The authors study the distinction between failing to retrieve an update and failing to use an update that was retrieved, on the single-hop FactConsolidation task of MemoryAgentBench.
We examine this distinction on the single-hop FactConsolidation task of MemoryAgentBench.
Source-stated
For Qwen2.5-7B-Instruct on a 6K history, BM25 accuracy fell from 45% to 32% as the retrieved evidence budget grew from 64 to 512 tokens; a gold-assisted audit found the newest statement in every retrieved set.
With Qwen2.5-7B-Instruct on 6K-history BM25 accuracy drops from 45% to 32% when the retrieved evidence budget grows from 64 to 512 tokens, even though a gold-assisted post-hoc audit finds the newest statement in every retrieved set.
Source-stated
With equal four-statement evidence size, replacing an unrelated statement with an obsolete competing statement reduced accuracy for both tested models; the authors classified all 50 harmful paired flips as stale.
replacing one unrelated statement with the obsolete competing statement reduces accuracy from 100% to 44.6% for Qwen2.5 and from 100% to 87.8% for Qwen3-14B, with all 50 harmful paired flips classified as stale.
Source-stated
In a 32K-history replication, BM25-64 performed better than BM25-512 in the reported results.
A 32K-history replication likewise favors BM25-64 over BM25-512 (69% versus 58%).
Source and contributor context
Poikarayil, Oleeviya Babu, Babu, Tony Basil, Nahhas, Abdulrahman, Mehta, Het Darshan, Baktash, Jawid Ahmad, De Luca, Ernesto William, Turowski, Klaus · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
AI agent
An AI system that chooses and carries out actions toward a goal, often using software tools.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: efa87b36-f8a0-4b49-8425-7fe7d53c2f66.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-05T15:02:30.502Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: useful_perspective.
Observation: 2d623088f7bd8d21fddddd3a6f7769e0c4aa387c212a8123298f4af54a836106. Source text hash: 8c65148a8958e293f26fcf49fc9db302353d224ce963130d1b5ee1d6668f16d2.
Revision and source-check history
Version 1 · 2026-11-05T15:02:30.502Z
When Retrieval Is Not Enough: Obsolete Facts in Agent Memory
The authors report that retrieving an updated fact may not be enough when an ’s memory also contains an obsolete version. In the single-hop FactConsolidation task of MemoryAgentBench, they tested Qwen2.5-7B-Instruct using BM25, a method for retrieving text by matching terms in a query. Accuracy fell from 45% to 32% when the retrieved evidence budget increased from 64 to 512 tokens, despite a gold-assisted audit finding the newest statement in every retrieved set. The authors report that replacing an unrelated statement with an obsolete competing statement reduced accuracy from 100% to 44.6% for Qwen2.5 and from 100% to 87.8% for Qwen3-14B. Their 32K-history replication also favored BM25-64 over BM25-512. These results suggest that obsolete information can interfere with using an updated fact, rather than simply adding irrelevant context. The evidence comes from a specific task and model setup, so it does not establish how often this problem occurs in deployed agents. Would the same pattern appear in other memory tasks or with different ways of retrieving and presenting updated facts?
Original source
cs.AI · Article
Source details
Original source date: Oct 26, 2026 (2026-10-26).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-05T15:02:30.502Z. First observed: 2026-10-28T08:18:37.332Z. Retrieved: 2026-10-30T03:01:49.624Z.
Limits
The reported experiments concern the single-hop FactConsolidation task of MemoryAgentBench and specified model setups; the abstract does not establish whether the results generalize to other tasks or deployed agents. The audit that found the newest statement in each retrieved set was gold-assisted, so it does not show that an agent could identify the newest statement on its own.
Open question
Would the same pattern appear in other memory tasks or with different ways of retrieving and presenting updated facts?
Source-stated
The authors study the distinction between failing to retrieve an update and failing to use an update that was retrieved, on the single-hop FactConsolidation task of MemoryAgentBench.
We examine this distinction on the single-hop FactConsolidation task of MemoryAgentBench.
Source-stated
For Qwen2.5-7B-Instruct on a 6K history, BM25 accuracy fell from 45% to 32% as the retrieved evidence budget grew from 64 to 512 tokens; a gold-assisted audit found the newest statement in every retrieved set.
With Qwen2.5-7B-Instruct on 6K-history BM25 accuracy drops from 45% to 32% when the retrieved evidence budget grows from 64 to 512 tokens, even though a gold-assisted post-hoc audit finds the newest statement in every retrieved set.
Source-stated
With equal four-statement evidence size, replacing an unrelated statement with an obsolete competing statement reduced accuracy for both tested models; the authors classified all 50 harmful paired flips as stale.
replacing one unrelated statement with the obsolete competing statement reduces accuracy from 100% to 44.6% for Qwen2.5 and from 100% to 87.8% for Qwen3-14B, with all 50 harmful paired flips classified as stale.
Source-stated
In a 32K-history replication, BM25-64 performed better than BM25-512 in the reported results.
A 32K-history replication likewise favors BM25-64 over BM25-512 (69% versus 58%).
Source and contributor context
Poikarayil, Oleeviya Babu, Babu, Tony Basil, Nahhas, Abdulrahman, Mehta, Het Darshan, Baktash, Jawid Ahmad, De Luca, Ernesto William, Turowski, Klaus · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
AI agent
An AI system that chooses and carries out actions toward a goal, often using software tools.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: efa87b36-f8a0-4b49-8425-7fe7d53c2f66.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-05T15:02:30.502Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: useful_perspective.
Observation: 2d623088f7bd8d21fddddd3a6f7769e0c4aa387c212a8123298f4af54a836106. Source text hash: 8c65148a8958e293f26fcf49fc9db302353d224ce963130d1b5ee1d6668f16d2.
passed · 2026-11-05T15:02:30.502Z · anthropic/claude-opus-5.5
Publication: efa87b36-f8a0-4b49-8425-7fe7d53c2f66
attribution — pass: The findings are attributed to the paper's authors ('The authors report'), and the arXiv listing names the authors. The interpretive sentence is framed as what the results 'suggest,' which matches the abstract's own hedged wording.
These results suggest that retrieving the correct fact is not enough.
summary — pass: The reading leads with the authors' central argument: retrieving an update is not enough when an obsolete version competes with it. It explains BM25 in plain language and reports figures that match the abstract. It also keeps the scope limits. Terms such as 'tokens,' 'single-hop' and 'gold-assisted' could use term-library tooltips, but they do not block understanding of the main point. The question follows from the evidence and is not leading.
An obsolete competing statement can be more harmful than an unrelated piece of evidence of the same size.
date — pass: originalPublished 2026-10-26 matches the arXiv v1 submission date shown on the page. The feed date of 2026-10-29 is a later announcement and is correctly not used.
[Submitted on 26 Oct 2026]
support — pass: Each claim's excerpt is verbatim from the abstract and supports the claim. The limitations are cautious and supported: the experiments are confined to the named task and models, and the audit was gold-assisted. The reading adds no unsupported benefits, costs or stakeholder claims. One minor omission: the reading drops the abstract's 'frozen prompt' condition for the substitution test. This does not misstate the result.
Under a frozen prompt and equal four-statement evidence size, replacing one unrelated statement with the obsolete competing statement reduces accuracy
media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source and the easiest way for readers to access the work.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
An AI system that chooses and carries out actions toward a goal, often using software tools.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivAn AI system that chooses and carries out actions toward a goal, often using software tools.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv