# When Retrieval Is Not Enough: Obsolete Facts in Agent Memory [Canonical reading](https://escapemeridian.com/readings/when-retrieval-is-not-enough-obsolete-facts-in-agent-memory-7a5ee2ca) The authors report that retrieving an updated fact may not be enough when an AI agent’s memory also contains an obsolete version. In the single-hop FactConsolidation task of MemoryAgentBench, they tested Qwen2.5-7B-Instruct using BM25, a method for retrieving text by matching terms in a query. Accuracy fell from 45% to 32% when the retrieved evidence budget increased from 64 to 512 tokens, despite a gold-assisted audit finding the newest statement in every retrieved set. The authors report that replacing an unrelated statement with an obsolete competing statement reduced accuracy from 100% to 44.6% for Qwen2.5 and from 100% to 87.8% for Qwen3-14B. Their 32K-history replication also favored BM25-64 over BM25-512. These results suggest that obsolete information can interfere with using an updated fact, rather than simply adding irrelevant context. The evidence comes from a specific benchmark task and model setup, so it does not establish how often this problem occurs in deployed agents. Would the same pattern appear in other memory tasks or with different ways of retrieving and presenting updated facts? Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Oct 26, 2026 (2026-10-26). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-05T15:02:30.502Z. First observed: 2026-10-28T08:18:37.332Z. Retrieved: 2026-10-30T03:01:49.624Z. - [When Retrieval Is Not Enough: Obsolete Facts in Agent Memory](https://arxiv.org/abs/2610.36308) ## Limits The reported experiments concern the single-hop FactConsolidation task of MemoryAgentBench and specified model setups; the abstract does not establish whether the results generalize to other tasks or deployed agents. The audit that found the newest statement in each retrieved set was gold-assisted, so it does not show that an agent could identify the newest statement on its own. ## Open question Would the same pattern appear in other memory tasks or with different ways of retrieving and presenting updated facts? ## Source-stated The authors study the distinction between failing to retrieve an update and failing to use an update that was retrieved, on the single-hop FactConsolidation task of MemoryAgentBench. > We examine this distinction on the single-hop FactConsolidation task of MemoryAgentBench. - [Supporting source](https://arxiv.org/abs/2610.36308) ## Source-stated For Qwen2.5-7B-Instruct on a 6K history, BM25 accuracy fell from 45% to 32% as the retrieved evidence budget grew from 64 to 512 tokens; a gold-assisted audit found the newest statement in every retrieved set. > With Qwen2.5-7B-Instruct on 6K-history BM25 accuracy drops from 45% to 32% when the retrieved evidence budget grows from 64 to 512 tokens, even though a gold-assisted post-hoc audit finds the newest statement in every retrieved set. - [Supporting source](https://arxiv.org/abs/2610.36308) ## Source-stated With equal four-statement evidence size, replacing an unrelated statement with an obsolete competing statement reduced accuracy for both tested models; the authors classified all 50 harmful paired flips as stale. > replacing one unrelated statement with the obsolete competing statement reduces accuracy from 100% to 44.6% for Qwen2.5 and from 100% to 87.8% for Qwen3-14B, with all 50 harmful paired flips classified as stale. - [Supporting source](https://arxiv.org/abs/2610.36308) ## Source-stated In a 32K-history replication, BM25-64 performed better than BM25-512 in the reported results. > A 32K-history replication likewise favors BM25-64 over BM25-512 (69% versus 58%). - [Supporting source](https://arxiv.org/abs/2610.36308) ## Source and contributor context Poikarayil, Oleeviya Babu, Babu, Tony Basil, Nahhas, Abdulrahman, Mehta, Het Darshan, Baktash, Jawid Ahmad, De Luca, Ernesto William, Turowski, Klaus · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## AI agent An AI system that chooses and carries out actions toward a goal, often using software tools. - [Definition reference](https://www.ibm.com/think/topics/ai-agents) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: efa87b36-f8a0-4b49-8425-7fe7d53c2f66. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-05T15:02:30.502Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: useful\_perspective. Observation: 2d623088f7bd8d21fddddd3a6f7769e0c4aa387c212a8123298f4af54a836106. Source text hash: 8c65148a8958e293f26fcf49fc9db302353d224ce963130d1b5ee1d6668f16d2. ## Revision and source-check history ### Version 1 — 2026-11-05T15:02:30.502Z The authors report that retrieving an updated fact may not be enough when an AI agent’s memory also contains an obsolete version. In the single-hop FactConsolidation task of MemoryAgentBench, they tested Qwen2.5-7B-Instruct using BM25, a method for retrieving text by matching terms in a query. Accuracy fell from 45% to 32% when the retrieved evidence budget increased from 64 to 512 tokens, despite a gold-assisted audit finding the newest statement in every retrieved set. The authors report that replacing an unrelated statement with an obsolete competing statement reduced accuracy from 100% to 44.6% for Qwen2.5 and from 100% to 87.8% for Qwen3-14B. Their 32K-history replication also favored BM25-64 over BM25-512. These results suggest that obsolete information can interfere with using an updated fact, rather than simply adding irrelevant context. The evidence comes from a specific benchmark task and model setup, so it does not establish how often this problem occurs in deployed agents. Would the same pattern appear in other memory tasks or with different ways of retrieving and presenting updated facts? ## Original source arXiv cs.AI · Article Original source date: Oct 26, 2026 (2026-10-26). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-05T15:02:30.502Z. First observed: 2026-10-28T08:18:37.332Z. Retrieved: 2026-10-30T03:01:49.624Z. - [When Retrieval Is Not Enough: Obsolete Facts in Agent Memory](https://arxiv.org/abs/2610.36308) ## Limits The reported experiments concern the single-hop FactConsolidation task of MemoryAgentBench and specified model setups; the abstract does not establish whether the results generalize to other tasks or deployed agents. The audit that found the newest statement in each retrieved set was gold-assisted, so it does not show that an agent could identify the newest statement on its own. ## Open question Would the same pattern appear in other memory tasks or with different ways of retrieving and presenting updated facts? ## Source-stated The authors study the distinction between failing to retrieve an update and failing to use an update that was retrieved, on the single-hop FactConsolidation task of MemoryAgentBench. > We examine this distinction on the single-hop FactConsolidation task of MemoryAgentBench. - [Supporting source](https://arxiv.org/abs/2610.36308) ## Source-stated For Qwen2.5-7B-Instruct on a 6K history, BM25 accuracy fell from 45% to 32% as the retrieved evidence budget grew from 64 to 512 tokens; a gold-assisted audit found the newest statement in every retrieved set. > With Qwen2.5-7B-Instruct on 6K-history BM25 accuracy drops from 45% to 32% when the retrieved evidence budget grows from 64 to 512 tokens, even though a gold-assisted post-hoc audit finds the newest statement in every retrieved set. - [Supporting source](https://arxiv.org/abs/2610.36308) ## Source-stated With equal four-statement evidence size, replacing an unrelated statement with an obsolete competing statement reduced accuracy for both tested models; the authors classified all 50 harmful paired flips as stale. > replacing one unrelated statement with the obsolete competing statement reduces accuracy from 100% to 44.6% for Qwen2.5 and from 100% to 87.8% for Qwen3-14B, with all 50 harmful paired flips classified as stale. - [Supporting source](https://arxiv.org/abs/2610.36308) ## Source-stated In a 32K-history replication, BM25-64 performed better than BM25-512 in the reported results. > A 32K-history replication likewise favors BM25-64 over BM25-512 (69% versus 58%). - [Supporting source](https://arxiv.org/abs/2610.36308) ## Source and contributor context Poikarayil, Oleeviya Babu, Babu, Tony Basil, Nahhas, Abdulrahman, Mehta, Het Darshan, Baktash, Jawid Ahmad, De Luca, Ernesto William, Turowski, Klaus · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## AI agent An AI system that chooses and carries out actions toward a goal, often using software tools. - [Definition reference](https://www.ibm.com/think/topics/ai-agents) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: efa87b36-f8a0-4b49-8425-7fe7d53c2f66. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-05T15:02:30.502Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: useful\_perspective. Observation: 2d623088f7bd8d21fddddd3a6f7769e0c4aa387c212a8123298f4af54a836106. Source text hash: 8c65148a8958e293f26fcf49fc9db302353d224ce963130d1b5ee1d6668f16d2. ### passed — 2026-11-05T15:02:30.502Z — anthropic/claude-opus-5.5 Publication: efa87b36-f8a0-4b49-8425-7fe7d53c2f66 attribution — pass: The findings are attributed to the paper's authors ('The authors report'), and the arXiv listing names the authors. The interpretive sentence is framed as what the results 'suggest,' which matches the abstract's own hedged wording. > These results suggest that retrieving the correct fact is not enough. summary — pass: The reading leads with the authors' central argument: retrieving an update is not enough when an obsolete version competes with it. It explains BM25 in plain language and reports figures that match the abstract. It also keeps the scope limits. Terms such as 'tokens,' 'single-hop' and 'gold-assisted' could use term-library tooltips, but they do not block understanding of the main point. The question follows from the evidence and is not leading. > An obsolete competing statement can be more harmful than an unrelated piece of evidence of the same size. date — pass: originalPublished 2026-10-26 matches the arXiv v1 submission date shown on the page. The feed date of 2026-10-29 is a later announcement and is correctly not used. > \[Submitted on 26 Oct 2026\] support — pass: Each claim's excerpt is verbatim from the abstract and supports the claim. The limitations are cautious and supported: the experiments are confined to the named task and models, and the audit was gold-assisted. The reading adds no unsupported benefits, costs or stakeholder claims. One minor omission: the reading drops the abstract's 'frozen prompt' condition for the substitution test. This does not misstate the result. > Under a frozen prompt and equal four-statement evidence size, replacing one unrelated statement with the obsolete competing statement reduces accuracy media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source and the easiest way for readers to access the work. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/7a5ee2ca46e6624c718559e42c2c1ddd9fe296fb53ed85e053218851e73ad651/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/when-retrieval-is-not-enough-obsolete-facts-in-agent-memory-7a5ee2ca.json)