Escape Meridian

Roush, Allen G., Nguyen, Minh Nhat, Shwartz-Ziv, Ravid, Goldfeder, Judah, Basu, Sanjay · Nov 17, 2026

Long-Context Generation Is a Sampling Problem (When You Turn Up the Heat)

Markdown · JSON · Original source

The authors argue that, at high temperature, the sampler—the rule that chooses the next token from a model’s probabilities—can determine whether long generation holds up, rather than more training being the only answer. They tested 29 on ten and needle retrieval out to one , including coding agents, long-form writing and long-context reading. In their results, samplers that keep the candidate-token pool small and steady as temperature rises retained accuracy; those admitting low-probability tokens lost it. At temperature 2, the authors report that coding agents using top-p, top-k or Top-H resolved none of 100 GitHub issues, while top-nsigma resolved 46 to 62 on three Qwen3.5 models. Across benchmarks, top-nsigma retained a median 100% of its temperature-1 score, compared with 26% for top-p. The authors link this difference to pool size: in one Llama-3.1-8B story, top-p’s pool grew from 4 tokens at temperature 1 to 51,845 at temperature 2, while top-nsigma stayed at 3. They also report that fixing the pool at temperature 1 stopped Top-H, min-p and p-less from collapsing when temperature was applied afterward. At temperature 1, most samplers performed about the same. The evidence presented here is limited to the tested models and tasks; whether the pattern extends beyond them remains open.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

arXiv cs.AIArticle · arxiv.orgOpen original source

Original source

cs.AI · Article

Source details

Original source date: Nov 17, 2026 (2026-11-17).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-18T15:01:34.080Z. First observed: 2026-11-18T05:13:57.783Z. Retrieved: 2026-11-18T14:49:27.375Z.

Referenced by

Recorded citations in public readings. A reference alone does not establish agreement or influence.

No public references recorded here yet.

Reference data (JSON)

Limits

The abstract reports results from 29 and the specified and tasks; it does not establish whether the findings extend beyond those tests. The supplied evidence is the abstract and does not provide detailed evaluation methods or results for each benchmark.

Open question

Do these results hold for and generation tasks beyond the ones tested?

Source-stated

The authors argue that sampler choice at high temperature can determine whether long generation holds up.

At high temperature, The sampler, i.e. the rule that picks the next token out of the model's probabilities, decides whether long and agentic generation survives and/or thrives.

Source-stated

The authors tested 29 open-weight models across ten benchmarks and needle retrieval out to one million tokens.

We test 29 open-weight models from 1B to 122B parameters on ten established benchmarks plus needle retrieval out to one million tokens

Source-stated

At temperature 2, the authors report a large difference in coding-agent results between the named samplers on three Qwen3.5 models.

At temperature 2, coding agents running top-p, top-k or Top-H resolve 0 of 100 GitHub issues, while the same agents running top-nsigma resolve 46 to 62

Source-stated

The authors report that candidate-pool size predicts judged quality across the tested sampler settings and temperatures.

Across 133 combinations of sampler setting and temperature, the typical pool size predicts judged quality (Spearman correlation of -0.77)

Source-stated

At temperature 1, most samplers performed about the same, according to the authors.

At the default temperature of 1 most samplers perform about the same

Source and contributor context

Roush, Allen G., Nguyen, Minh Nhat, Shwartz-Ziv, Ravid, Goldfeder, Judah, Basu, Sanjay · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

No suitable media was found by the supported page extraction.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Open weights

A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.

Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

Model tokens

Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens.

arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

Editorial provenance

Version 1. Publication ID: 2b216d3a-39c2-4db4-8a19-52343885f679.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-18T15:01:34.080Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material_development.

Observation: 94601fd6313cc467c10d028e746043694038b8ac6c6be2d042af996964da5a90. Source text hash: 746a140ba63c658c2115097dbfa87d8a74e774219ec84d18870a8add17c43bbf.

Revision and source-check history

Version 1 · 2026-11-18T15:01:34.080Z

Long-Context Generation Is a Sampling Problem (When You Turn Up the Heat)

The authors argue that, at high temperature, the sampler—the rule that chooses the next token from a model’s probabilities—can determine whether long generation holds up, rather than more training being the only answer. They tested 29 on ten and needle retrieval out to one , including coding agents, long-form writing and long-context reading. In their results, samplers that keep the candidate-token pool small and steady as temperature rises retained accuracy; those admitting low-probability tokens lost it. At temperature 2, the authors report that coding agents using top-p, top-k or Top-H resolved none of 100 GitHub issues, while top-nsigma resolved 46 to 62 on three Qwen3.5 models. Across benchmarks, top-nsigma retained a median 100% of its temperature-1 score, compared with 26% for top-p. The authors link this difference to pool size: in one Llama-3.1-8B story, top-p’s pool grew from 4 tokens at temperature 1 to 51,845 at temperature 2, while top-nsigma stayed at 3. They also report that fixing the pool at temperature 1 stopped Top-H, min-p and p-less from collapsing when temperature was applied afterward. At temperature 1, most samplers performed about the same. The evidence presented here is limited to the tested models and tasks; whether the pattern extends beyond them remains open.

Original source

cs.AI · Article

Source details

Original source date: Nov 17, 2026 (2026-11-17).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-18T15:01:34.080Z. First observed: 2026-11-18T05:13:57.783Z. Retrieved: 2026-11-18T14:49:27.375Z.

Limits

The abstract reports results from 29 and the specified and tasks; it does not establish whether the findings extend beyond those tests. The supplied evidence is the abstract and does not provide detailed evaluation methods or results for each benchmark.

Open question

Do these results hold for and generation tasks beyond the ones tested?

Source-stated

The authors argue that sampler choice at high temperature can determine whether long generation holds up.

At high temperature, The sampler, i.e. the rule that picks the next token out of the model's probabilities, decides whether long and agentic generation survives and/or thrives.

Source-stated

The authors tested 29 open-weight models across ten benchmarks and needle retrieval out to one million tokens.

We test 29 open-weight models from 1B to 122B parameters on ten established benchmarks plus needle retrieval out to one million tokens

Source-stated

At temperature 2, the authors report a large difference in coding-agent results between the named samplers on three Qwen3.5 models.

At temperature 2, coding agents running top-p, top-k or Top-H resolve 0 of 100 GitHub issues, while the same agents running top-nsigma resolve 46 to 62

Source-stated

The authors report that candidate-pool size predicts judged quality across the tested sampler settings and temperatures.

Across 133 combinations of sampler setting and temperature, the typical pool size predicts judged quality (Spearman correlation of -0.77)

Source-stated

At temperature 1, most samplers performed about the same, according to the authors.

At the default temperature of 1 most samplers perform about the same

Source and contributor context

Roush, Allen G., Nguyen, Minh Nhat, Shwartz-Ziv, Ravid, Goldfeder, Judah, Basu, Sanjay · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

No suitable media was found by the supported page extraction.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Open weights

A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.

Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

Model tokens

Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens.

arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

Editorial provenance

Version 1. Publication ID: 2b216d3a-39c2-4db4-8a19-52343885f679.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-18T15:01:34.080Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material_development.

Observation: 94601fd6313cc467c10d028e746043694038b8ac6c6be2d042af996964da5a90. Source text hash: 746a140ba63c658c2115097dbfa87d8a74e774219ec84d18870a8add17c43bbf.

uncertain · 2026-11-18T15:01:34.080Z · anthropic/claude-opus-5.5

Publication: 2b216d3a-39c2-4db4-8a19-52343885f679

attribution — pass: The reading consistently attributes the argument and results to the paper's authors, and the arXiv source lists them. It does not present their findings as Escape Meridian's conclusions.

Authors:Allen G. Roush, Minh Nhat Nguyen, Ravid Shwartz-Ziv, Judah Goldfeder, Sanjay Basu

summary — uncertain: The reading drops the problem the paper starts from: open-weight models are widely observed to get worse the longer they generate. Without it, 'whether long generation holds up' lacks a clear stake. 'Temperature' is central to the paper and its title but is never explained, so a general reader cannot follow why raising it enlarges the candidate pool. Smallest repair: open with the degradation problem, and give a plain-language gloss or tooltip for temperature as the setting that controls randomness. A tooltip for top-p or top-nsigma would also help. 'Lost it' softens the source's 'fall apart'.

Open-weight language models are widely observed to get worse the longer they generate.

date — pass: The displayed original date is 2026-11-17, which matches the v1 submission date. The feed date of 2026-11-18 is later and is correctly not used.

[Submitted on 17 Nov 2026]

support — pass: The figures in the reading match the abstract: 29 models, ten benchmarks, 0 of 100 versus 46 to 62 issues resolved, median 100% versus 26%, pool sizes of 4 to 51,845 versus 3, Spearman correlation of -0.77, and the fix of choosing the pool at temperature 1. The phrase 'can determine' is a cautious rendering of the source's 'decides'. The limitations and the closing question do not overreach beyond the evidence.

Over all benchmarks top-nsigma keeps a median 100% of its temperature-1 score where top-p keeps 26%.

media — pass: The selected media is the verified arXiv abstract page itself, so it is the direct original source and the easiest route to the paper.

Challenge or correct this reading

Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.