# The Silent Cost of Conversation History in LLM Reasoning

[Canonical reading](https://escapemeridian.com/readings/the-silent-cost-of-conversation-history-in-llm-reasoning-d47f1c2a)

Ali Şenol’s paper asks whether an LLM’s accumulated conversation history can lower its accuracy, and whether process-level quality metrics can warn before that happens. In Study 1, retaining earlier replies lowered final success on 48 sharded math tasks for GPT-4o-mini and Mistral-7B; Llama-4 Scout showed the same direction, but the result was not significant. In Study 2, accuracy with the full history fell from about 0.6 to 0.1 within twenty turns at matched difficulty. A solver-verified state summary kept accuracy 0.52-0.59 higher. Yet consistency and logical coherence fell far less than accuracy, and stability rose. The paper reports that none of the tested metrics gave robust, model-general early warning; in Study 2, repeat-based metrics were complete only for GPT-4o-mini. The findings point to a practical mismatch: conversation signals can look relatively steady even as task accuracy drops. Şenol recommends keeping task state outside the transcript and monitoring across sessions. The abstract reports results from these studies, while noting that the tested monitors had power in a single 100-turn stream only for very large drops.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

## Original source

arXiv cs.AI · Article

Original source date: Nov 15, 2026 (2026-11-15).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-17T08:48:00.277Z. First observed: 2026-11-17T08:33:31.529Z. Retrieved: 2026-11-17T08:47:06.243Z.

- [The Silent Cost of Conversation History in LLM Reasoning](https://arxiv.org/abs/2611.20071)

## Limits

The abstract reports results from two studies, and does not establish whether the findings generalize beyond the tested tasks and models. In Study 2, the paper says repeat-based metrics were complete only for GPT-4o-mini; the tested monitors in a single 100-turn stream had power only for very large drops.


## Open question

What conditions, if any, would let the tested process-level metrics give robust, model-general early warning of accuracy loss?


## Source-stated

In Study 1, retaining earlier replies lowered final success for GPT-4o-mini and Mistral-7B on the tested sharded math tasks; Llama-4 Scout showed the same direction, but not significantly.

> retaining its replies lowered final success by 0.21 for GPT-4o-mini and 0.29 for Mistral-7B (Holm-adjusted p = 0.026 and 0.004); Llama-4 Scout showed the same direction, not significant.

- [Supporting source](https://arxiv.org/abs/2611.20071)

## Source-stated

In Study 2, with full history, accuracy fell substantially within twenty turns at matched difficulty, while a solver-verified state summary kept it higher.

> With the full history, accuracy fell from about 0.6 to 0.1 within twenty turns at matched difficulty, whereas a solver-verified state summary kept it 0.52-0.59 higher

- [Supporting source](https://arxiv.org/abs/2611.20071)

## Source-stated

The paper reports that tested process-level measures did not give robust, model-general early warning, and that the single-stream monitors had power only for very large drops.

> None of these metrics, a paired readiness measure or calibrated single-stream drift detectors gave robust, model-general early warning

- [Supporting source](https://arxiv.org/abs/2611.20071)

## Source-stated

Şenol recommends keeping task state outside the transcript and monitoring across sessions.

> Systems built on LLMs should keep task state outside the transcript and monitor across sessions.

- [Supporting source](https://arxiv.org/abs/2611.20071)

## Source and contributor context

Şenol, Ali · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Large language model

A language model with many learned numerical settings, used to process and generate language.

- [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: 7565d5ce-f95f-438d-8b98-4a62d6e70853.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-17T08:48:00.277Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: ebb3188d9e9ac6bf717a8a651f159f04a4c329a679df3d80950dcd2ff907a75b. Source text hash: bd97980a07a05ffa4a3b47b1b8cd3de94e079d745e67cf8cb828f22fd29ff96a.


## Revision and source-check history

### Version 1 — 2026-11-17T08:48:00.277Z

Ali Şenol’s paper asks whether an LLM’s accumulated conversation history can lower its accuracy, and whether process-level quality metrics can warn before that happens. In Study 1, retaining earlier replies lowered final success on 48 sharded math tasks for GPT-4o-mini and Mistral-7B; Llama-4 Scout showed the same direction, but the result was not significant. In Study 2, accuracy with the full history fell from about 0.6 to 0.1 within twenty turns at matched difficulty. A solver-verified state summary kept accuracy 0.52-0.59 higher. Yet consistency and logical coherence fell far less than accuracy, and stability rose. The paper reports that none of the tested metrics gave robust, model-general early warning; in Study 2, repeat-based metrics were complete only for GPT-4o-mini. The findings point to a practical mismatch: conversation signals can look relatively steady even as task accuracy drops. Şenol recommends keeping task state outside the transcript and monitoring across sessions. The abstract reports results from these studies, while noting that the tested monitors had power in a single 100-turn stream only for very large drops.

## Original source

arXiv cs.AI · Article

Original source date: Nov 15, 2026 (2026-11-15).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-17T08:48:00.277Z. First observed: 2026-11-17T08:33:31.529Z. Retrieved: 2026-11-17T08:47:06.243Z.

- [The Silent Cost of Conversation History in LLM Reasoning](https://arxiv.org/abs/2611.20071)

## Limits

The abstract reports results from two studies, and does not establish whether the findings generalize beyond the tested tasks and models. In Study 2, the paper says repeat-based metrics were complete only for GPT-4o-mini; the tested monitors in a single 100-turn stream had power only for very large drops.


## Open question

What conditions, if any, would let the tested process-level metrics give robust, model-general early warning of accuracy loss?


## Source-stated

In Study 1, retaining earlier replies lowered final success for GPT-4o-mini and Mistral-7B on the tested sharded math tasks; Llama-4 Scout showed the same direction, but not significantly.

> retaining its replies lowered final success by 0.21 for GPT-4o-mini and 0.29 for Mistral-7B (Holm-adjusted p = 0.026 and 0.004); Llama-4 Scout showed the same direction, not significant.

- [Supporting source](https://arxiv.org/abs/2611.20071)

## Source-stated

In Study 2, with full history, accuracy fell substantially within twenty turns at matched difficulty, while a solver-verified state summary kept it higher.

> With the full history, accuracy fell from about 0.6 to 0.1 within twenty turns at matched difficulty, whereas a solver-verified state summary kept it 0.52-0.59 higher

- [Supporting source](https://arxiv.org/abs/2611.20071)

## Source-stated

The paper reports that tested process-level measures did not give robust, model-general early warning, and that the single-stream monitors had power only for very large drops.

> None of these metrics, a paired readiness measure or calibrated single-stream drift detectors gave robust, model-general early warning

- [Supporting source](https://arxiv.org/abs/2611.20071)

## Source-stated

Şenol recommends keeping task state outside the transcript and monitoring across sessions.

> Systems built on LLMs should keep task state outside the transcript and monitor across sessions.

- [Supporting source](https://arxiv.org/abs/2611.20071)

## Source and contributor context

Şenol, Ali · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Large language model

A language model with many learned numerical settings, used to process and generate language.

- [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: 7565d5ce-f95f-438d-8b98-4a62d6e70853.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-17T08:48:00.277Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: ebb3188d9e9ac6bf717a8a651f159f04a4c329a679df3d80950dcd2ff907a75b. Source text hash: bd97980a07a05ffa4a3b47b1b8cd3de94e079d745e67cf8cb828f22fd29ff96a.


### uncertain — 2026-11-17T08:48:00.277Z — anthropic/claude-opus-5.5

Publication: 7565d5ce-f95f-438d-8b98-4a62d6e70853

attribution — pass: The draft correctly attributes the paper and its recommendation to Ali Şenol, the arXiv author listed on the page. Source findings are framed as the paper's reports rather than as established facts.

> Authors:Ali Şenol

summary — uncertain: The summary is faithful, but central terms go unexplained for a general reader: LLM, 'sharded math tasks', 'solver-verified state summary', 'process-level quality metrics' and 'repeat-based metrics'. It also omits what Study 2 actually tested: CDR-Stream, a set of 100-turn state-tracking dialogues with an exact answer per turn. Without that, the 'twenty turns' result and the single-stream limit lack context. Smallest repair: briefly describe Study 2 and gloss these terms, or add tooltips for them.

> Study 2 introduces CDR-Stream, 100-turn state-tracking dialogues with an exact answer per turn and per-model difficulty calibration.

date — pass: originalPublished is 2026-11-15, which matches the arXiv v1 submission date. The later feed date of 2026-11-17 is correctly not used.

> \[Submitted on 15 Nov 2026\]

support — pass: The claims match the abstract's figures and conditions, including the non-significant Llama-4 Scout result, the matched-difficulty decline and the limits on early warning. One sentence is unlabeled interpretation but stays close to the data: 'conversation signals can look relatively steady even as task accuracy drops'. Consistency and coherence fell far less than accuracy and stability rose, so it is supported. Labeling it as an inference would be cleaner. The question follows from the evidence and is not leading.

> Consistency and logical coherence fell far less than accuracy, and stability rose.

media — pass: The selected media is the verified direct arXiv abstract page for this paper. It is the most direct way to access the source.

## Recorded references

Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter.

No public references recorded here yet.

[Reference data (JSON)](/readings/d47f1c2a1f5392eac4bd036ae73bd2428295fb04a4ce62f79f38a8f1625011fd/references.json)

## Corrections and challenges

[Send a correction or an opposing source](https://escapemeridian.com/corrections/)

[JSON record](https://escapemeridian.com/readings/the-silent-cost-of-conversation-history-in-llm-reasoning-d47f1c2a.json)
