Jayaram, K. R., Isahagian, Vatche, Aravindan, Aswath Vaithinathan, Muthusamy, Vinod, Oum, Punleuk, Fang, Gaodan, Ulewicz, Michal, Thomas, Gegi, Duesterwald, Evelyn · Nov 19, 2026
How Much Should an Agent Remember? Deployment Lessons from Self-Distilled Memory for LLM Agents
Markdown · JSON · Original source
The authors describe a pipeline that turns an agent’s own past task attempts into short lessons, then consolidates them into guidelines the agent can use later. Their central finding is that the useful amount of memory depends on the model: adding guidelines helped a strong model, while a weaker model did better with a selected subset than with all of them. On the AppWorld , DeepSeek-V3.2 gained 9.5 points in task completion and 16.1 points in scenario completion with its full set of 191 guidelines. For gpt-oss-120b, task completion rose from 39.9% to 51.2% with all 199 guidelines, but reached 55.4% on average across two runs with a selected subset of about 29. The authors also describe deployment failures and measurement pitfalls, including a guideline that backfired, provenance lost during consolidation, and an error that made every task fail while leaving a plausible log. These cases show why the team treats memory design and evaluation as practical deployment problems, not simply a matter of storing more guidance. The pipeline is available in the open-source altk-evolve library and is deployed in IBM’s CUGA agent and watsonx Orchestrate platform. The reported results use AppWorld as the primary instrument, so how well the findings transfer to other tasks remains unresolved.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 19, 2026 (2026-11-19).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-20T07:01:51.266Z. First observed: 2026-11-20T05:07:45.173Z. Retrieved: 2026-11-20T07:01:02.040Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The reported results use AppWorld as the primary instrument; this evidence does not establish whether the findings transfer to other tasks. The abstract reports that the selected-subset result for gpt-oss-120b is the mean of two runs.
Open question
What explains why gpt-oss-120b performed better with a selected subset of guidelines than with its full set?
Source-stated
The authors describe a pipeline that mines lessons from an agent’s past trajectories and consolidates them into guidelines for later use.
It first mines short \emph{lessons} from the agent's own trajectories, one to three per episode. It then consolidates those lessons into a smaller set of \emph{guidelines}
Source-stated
The authors report that a strong model improved with its full set of guidelines on AppWorld.
DeepSeek-V3.2 gains +9.5 points of task completion and +16.1 points of scenario completion with its full 191-guideline set.
Source-stated
For gpt-oss-120b, the authors report higher task completion with a selected subset than with all guidelines.
It reaches 51.2\% with all 199 of its guidelines but 55.4\% (mean of two runs) with a selected subset of about 29.
Source-stated
The authors report deployment and evaluation problems, including lost provenance and an API error that caused every task to fail while producing a plausible log.
provenance silently dropped during consolidation, a swallowed API error that failed every task while producing a plausible log
Source-stated
The authors say the pipeline is included in an open-source library and deployed in two IBM offerings.
The pipeline ships in the open-source altk-evolve library and is deployed in IBM's CUGA agent and watsonx Orchestrate platform.
Escape Meridian interpretation
The authors identify the AppWorld benchmark as their primary instrument, limiting what these results establish about other settings.
Our primary instrument is the AppWorld benchmark
Source and contributor context
Jayaram, K. R., Isahagian, Vatche, Aravindan, Aswath Vaithinathan, Muthusamy, Vinod, Oum, Punleuk, Fang, Gaodan, Ulewicz, Michal, Thomas, Gegi, Duesterwald, Evelyn · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
API
An application programming interface: a defined way for software to request data or actions from another system.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: ad30a210-b0ed-468b-84f3-9a2fc418aaf3.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-20T07:01:51.266Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 517d12dd03b4ec8d74464464d90bc994017dfa922852e626dd6aae32e0fb38d3. Source text hash: 35a2bc4eb3d9397af70db78279e483316984b2da41f5033cb881ba74d9c96f65.
Revision and source-check history
Version 1 · 2026-11-20T07:01:51.266Z
How Much Should an Agent Remember? Deployment Lessons from Self-Distilled Memory for LLM Agents
The authors describe a pipeline that turns an agent’s own past task attempts into short lessons, then consolidates them into guidelines the agent can use later. Their central finding is that the useful amount of memory depends on the model: adding guidelines helped a strong model, while a weaker model did better with a selected subset than with all of them. On the AppWorld , DeepSeek-V3.2 gained 9.5 points in task completion and 16.1 points in scenario completion with its full set of 191 guidelines. For gpt-oss-120b, task completion rose from 39.9% to 51.2% with all 199 guidelines, but reached 55.4% on average across two runs with a selected subset of about 29. The authors also describe deployment failures and measurement pitfalls, including a guideline that backfired, provenance lost during consolidation, and an error that made every task fail while leaving a plausible log. These cases show why the team treats memory design and evaluation as practical deployment problems, not simply a matter of storing more guidance. The pipeline is available in the open-source altk-evolve library and is deployed in IBM’s CUGA agent and watsonx Orchestrate platform. The reported results use AppWorld as the primary instrument, so how well the findings transfer to other tasks remains unresolved.
Original source
cs.AI · Article
Source details
Original source date: Nov 19, 2026 (2026-11-19).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-20T07:01:51.266Z. First observed: 2026-11-20T05:07:45.173Z. Retrieved: 2026-11-20T07:01:02.040Z.
Limits
The reported results use AppWorld as the primary instrument; this evidence does not establish whether the findings transfer to other tasks. The abstract reports that the selected-subset result for gpt-oss-120b is the mean of two runs.
Open question
What explains why gpt-oss-120b performed better with a selected subset of guidelines than with its full set?
Source-stated
The authors describe a pipeline that mines lessons from an agent’s past trajectories and consolidates them into guidelines for later use.
It first mines short \emph{lessons} from the agent's own trajectories, one to three per episode. It then consolidates those lessons into a smaller set of \emph{guidelines}
Source-stated
The authors report that a strong model improved with its full set of guidelines on AppWorld.
DeepSeek-V3.2 gains +9.5 points of task completion and +16.1 points of scenario completion with its full 191-guideline set.
Source-stated
For gpt-oss-120b, the authors report higher task completion with a selected subset than with all guidelines.
It reaches 51.2\% with all 199 of its guidelines but 55.4\% (mean of two runs) with a selected subset of about 29.
Source-stated
The authors report deployment and evaluation problems, including lost provenance and an API error that caused every task to fail while producing a plausible log.
provenance silently dropped during consolidation, a swallowed API error that failed every task while producing a plausible log
Source-stated
The authors say the pipeline is included in an open-source library and deployed in two IBM offerings.
The pipeline ships in the open-source altk-evolve library and is deployed in IBM's CUGA agent and watsonx Orchestrate platform.
Escape Meridian interpretation
The authors identify the AppWorld benchmark as their primary instrument, limiting what these results establish about other settings.
Our primary instrument is the AppWorld benchmark
Source and contributor context
Jayaram, K. R., Isahagian, Vatche, Aravindan, Aswath Vaithinathan, Muthusamy, Vinod, Oum, Punleuk, Fang, Gaodan, Ulewicz, Michal, Thomas, Gegi, Duesterwald, Evelyn · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
API
An application programming interface: a defined way for software to request data or actions from another system.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: ad30a210-b0ed-468b-84f3-9a2fc418aaf3.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-20T07:01:51.266Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 517d12dd03b4ec8d74464464d90bc994017dfa922852e626dd6aae32e0fb38d3. Source text hash: 35a2bc4eb3d9397af70db78279e483316984b2da41f5033cb881ba74d9c96f65.
passed · 2026-11-20T07:01:51.266Z · anthropic/claude-opus-5.5
Publication: ad30a210-b0ed-468b-84f3-9a2fc418aaf3
attribution — pass: The reading attributes the pipeline, findings, failure cases and deployment statements to the paper's authors throughout. It does not present the benchmark results or deployment claims as independently verified facts.
We report what we learned building, measuring, and productizing it.
summary — pass: The reading opens with the actor and the development, glosses trajectories as past task attempts, and keeps the central thesis that the right amount of memory depends on the model. Figures match the abstract. 'Provenance' and AppWorld are not explained, but neither prevents a general reader from following the main point. The question addresses a real unresolved issue the abstract leaves open. Omitting the baseline-confound example is acceptable compression.
The central finding is that the right amount of memory is a property of the model, not the memory.
date — pass: The originalPublished value, 2026-11-19, matches the arXiv v1 submission date. It is not the later feed date of 2026-11-20.
[Submitted on 19 Nov 2026]
support — pass: Each claim is backed by an exact abstract excerpt. The one interpretive claim, that relying on AppWorld limits generalization, is labeled as interpretation and stated cautiously. The line saying the team treats memory as a practical deployment problem fits the authors' stated aim of distilling deployment recommendations. The reading invents no benefits or costs.
We distill these into recommendations for teams deploying agent memory.
media — pass: The selected media is the verified direct link to the arXiv abstract page, which is the original source. It is the most direct way for readers to access the paper.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A language model with many learned numerical settings, used to process and generate language.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceAn application programming interface: a defined way for software to request data or actions from another system.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA language model with many learned numerical settings, used to process and generate language.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceAn application programming interface: a defined way for software to request data or actions from another system.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv