# Long-Context Generation Is a Sampling Problem (When You Turn Up the Heat)

[Canonical reading](https://escapemeridian.com/readings/long-context-generation-is-a-sampling-problem-when-you-turn-up-b434e401)

The authors argue that, at high temperature, the sampler—the rule that chooses the next token from a model’s probabilities—can determine whether long generation holds up, rather than more training being the only answer. They tested 29 open-weight models on ten benchmarks and needle retrieval out to one million tokens, including coding agents, long-form writing and long-context reading. In their results, samplers that keep the candidate-token pool small and steady as temperature rises retained accuracy; those admitting low-probability tokens lost it. At temperature 2, the authors report that coding agents using top-p, top-k or Top-H resolved none of 100 GitHub issues, while top-nsigma resolved 46 to 62 on three Qwen3.5 models. Across benchmarks, top-nsigma retained a median 100% of its temperature-1 score, compared with 26% for top-p. The authors link this difference to pool size: in one Llama-3.1-8B story, top-p’s pool grew from 4 tokens at temperature 1 to 51,845 at temperature 2, while top-nsigma stayed at 3. They also report that fixing the pool at temperature 1 stopped Top-H, min-p and p-less from collapsing when temperature was applied afterward. At temperature 1, most samplers performed about the same. The evidence presented here is limited to the tested models and tasks; whether the pattern extends beyond them remains open.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

## Original source

arXiv cs.AI · Article

Original source date: Nov 17, 2026 (2026-11-17).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-18T15:01:34.080Z. First observed: 2026-11-18T05:13:57.783Z. Retrieved: 2026-11-18T14:49:27.375Z.

- [Long-Context Generation Is a Sampling Problem (When You Turn Up the Heat)](https://arxiv.org/abs/2611.22148)

## Limits

The abstract reports results from 29 open-weight models and the specified benchmarks and tasks; it does not establish whether the findings extend beyond those tests. The supplied evidence is the abstract and does not provide detailed evaluation methods or results for each benchmark.


## Open question

Do these results hold for open-weight models and generation tasks beyond the ones tested?


## Source-stated

The authors argue that sampler choice at high temperature can determine whether long generation holds up.

> At high temperature, The sampler, i.e. the rule that picks the next token out of the model's probabilities, decides whether long and agentic generation survives and/or thrives.

- [Supporting source](https://arxiv.org/abs/2611.22148)

## Source-stated

The authors tested 29 open-weight models across ten benchmarks and needle retrieval out to one million tokens.

> We test 29 open-weight models from 1B to 122B parameters on ten established benchmarks plus needle retrieval out to one million tokens

- [Supporting source](https://arxiv.org/abs/2611.22148)

## Source-stated

At temperature 2, the authors report a large difference in coding-agent results between the named samplers on three Qwen3.5 models.

> At temperature 2, coding agents running top-p, top-k or Top-H resolve 0 of 100 GitHub issues, while the same agents running top-nsigma resolve 46 to 62

- [Supporting source](https://arxiv.org/abs/2611.22148)

## Source-stated

The authors report that candidate-pool size predicts judged quality across the tested sampler settings and temperatures.

> Across 133 combinations of sampler setting and temperature, the typical pool size predicts judged quality (Spearman correlation of -0.77)

- [Supporting source](https://arxiv.org/abs/2611.22148)

## Source-stated

At temperature 1, most samplers performed about the same, according to the authors.

> At the default temperature of 1 most samplers perform about the same

- [Supporting source](https://arxiv.org/abs/2611.22148)

## Source and contributor context

Roush, Allen G., Nguyen, Minh Nhat, Shwartz-Ziv, Ravid, Goldfeder, Judah, Basu, Sanjay · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Open weights

A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.

- [Definition reference](https://opensource.org/ai/open-weights)

## Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

- [Definition reference](https://arxiv.org/abs/2211.09110)

## Model tokens

Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens.

- [Definition reference](https://developers.google.com/machine-learning/glossary#token)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: 2b216d3a-39c2-4db4-8a19-52343885f679.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-18T15:01:34.080Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 94601fd6313cc467c10d028e746043694038b8ac6c6be2d042af996964da5a90. Source text hash: 746a140ba63c658c2115097dbfa87d8a74e774219ec84d18870a8add17c43bbf.


## Revision and source-check history

### Version 1 — 2026-11-18T15:01:34.080Z

The authors argue that, at high temperature, the sampler—the rule that chooses the next token from a model’s probabilities—can determine whether long generation holds up, rather than more training being the only answer. They tested 29 open-weight models on ten benchmarks and needle retrieval out to one million tokens, including coding agents, long-form writing and long-context reading. In their results, samplers that keep the candidate-token pool small and steady as temperature rises retained accuracy; those admitting low-probability tokens lost it. At temperature 2, the authors report that coding agents using top-p, top-k or Top-H resolved none of 100 GitHub issues, while top-nsigma resolved 46 to 62 on three Qwen3.5 models. Across benchmarks, top-nsigma retained a median 100% of its temperature-1 score, compared with 26% for top-p. The authors link this difference to pool size: in one Llama-3.1-8B story, top-p’s pool grew from 4 tokens at temperature 1 to 51,845 at temperature 2, while top-nsigma stayed at 3. They also report that fixing the pool at temperature 1 stopped Top-H, min-p and p-less from collapsing when temperature was applied afterward. At temperature 1, most samplers performed about the same. The evidence presented here is limited to the tested models and tasks; whether the pattern extends beyond them remains open.

## Original source

arXiv cs.AI · Article

Original source date: Nov 17, 2026 (2026-11-17).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-18T15:01:34.080Z. First observed: 2026-11-18T05:13:57.783Z. Retrieved: 2026-11-18T14:49:27.375Z.

- [Long-Context Generation Is a Sampling Problem (When You Turn Up the Heat)](https://arxiv.org/abs/2611.22148)

## Limits

The abstract reports results from 29 open-weight models and the specified benchmarks and tasks; it does not establish whether the findings extend beyond those tests. The supplied evidence is the abstract and does not provide detailed evaluation methods or results for each benchmark.


## Open question

Do these results hold for open-weight models and generation tasks beyond the ones tested?


## Source-stated

The authors argue that sampler choice at high temperature can determine whether long generation holds up.

> At high temperature, The sampler, i.e. the rule that picks the next token out of the model's probabilities, decides whether long and agentic generation survives and/or thrives.

- [Supporting source](https://arxiv.org/abs/2611.22148)

## Source-stated

The authors tested 29 open-weight models across ten benchmarks and needle retrieval out to one million tokens.

> We test 29 open-weight models from 1B to 122B parameters on ten established benchmarks plus needle retrieval out to one million tokens

- [Supporting source](https://arxiv.org/abs/2611.22148)

## Source-stated

At temperature 2, the authors report a large difference in coding-agent results between the named samplers on three Qwen3.5 models.

> At temperature 2, coding agents running top-p, top-k or Top-H resolve 0 of 100 GitHub issues, while the same agents running top-nsigma resolve 46 to 62

- [Supporting source](https://arxiv.org/abs/2611.22148)

## Source-stated

The authors report that candidate-pool size predicts judged quality across the tested sampler settings and temperatures.

> Across 133 combinations of sampler setting and temperature, the typical pool size predicts judged quality (Spearman correlation of -0.77)

- [Supporting source](https://arxiv.org/abs/2611.22148)

## Source-stated

At temperature 1, most samplers performed about the same, according to the authors.

> At the default temperature of 1 most samplers perform about the same

- [Supporting source](https://arxiv.org/abs/2611.22148)

## Source and contributor context

Roush, Allen G., Nguyen, Minh Nhat, Shwartz-Ziv, Ravid, Goldfeder, Judah, Basu, Sanjay · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Open weights

A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.

- [Definition reference](https://opensource.org/ai/open-weights)

## Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

- [Definition reference](https://arxiv.org/abs/2211.09110)

## Model tokens

Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens.

- [Definition reference](https://developers.google.com/machine-learning/glossary#token)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: 2b216d3a-39c2-4db4-8a19-52343885f679.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-18T15:01:34.080Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 94601fd6313cc467c10d028e746043694038b8ac6c6be2d042af996964da5a90. Source text hash: 746a140ba63c658c2115097dbfa87d8a74e774219ec84d18870a8add17c43bbf.


### uncertain — 2026-11-18T15:01:34.080Z — anthropic/claude-opus-5.5

Publication: 2b216d3a-39c2-4db4-8a19-52343885f679

attribution — pass: The reading consistently attributes the argument and results to the paper's authors, and the arXiv source lists them. It does not present their findings as Escape Meridian's conclusions.

> Authors:Allen G. Roush, Minh Nhat Nguyen, Ravid Shwartz-Ziv, Judah Goldfeder, Sanjay Basu

summary — uncertain: The reading drops the problem the paper starts from: open-weight models are widely observed to get worse the longer they generate. Without it, 'whether long generation holds up' lacks a clear stake. 'Temperature' is central to the paper and its title but is never explained, so a general reader cannot follow why raising it enlarges the candidate pool. Smallest repair: open with the degradation problem, and give a plain-language gloss or tooltip for temperature as the setting that controls randomness. A tooltip for top-p or top-nsigma would also help. 'Lost it' softens the source's 'fall apart'.

> Open-weight language models are widely observed to get worse the longer they generate.

date — pass: The displayed original date is 2026-11-17, which matches the v1 submission date. The feed date of 2026-11-18 is later and is correctly not used.

> \[Submitted on 17 Nov 2026\]

support — pass: The figures in the reading match the abstract: 29 models, ten benchmarks, 0 of 100 versus 46 to 62 issues resolved, median 100% versus 26%, pool sizes of 4 to 51,845 versus 3, Spearman correlation of -0.77, and the fix of choosing the pool at temperature 1. The phrase 'can determine' is a cautious rendering of the source's 'decides'. The limitations and the closing question do not overreach beyond the evidence.

> Over all benchmarks top-nsigma keeps a median 100% of its temperature-1 score where top-p keeps 26%.

media — pass: The selected media is the verified arXiv abstract page itself, so it is the direct original source and the easiest route to the paper.

## Recorded references

Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter.

No public references recorded here yet.

[Reference data (JSON)](/readings/b434e4013d9220f4a526d55e0aedc92d27e22de9b7505e7937f1f6e821a7870a/references.json)

## Corrections and challenges

[Send a correction or an opposing source](https://escapemeridian.com/corrections/)

[JSON record](https://escapemeridian.com/readings/long-context-generation-is-a-sampling-problem-when-you-turn-up-b434e401.json)
