# Milios and Pal propose a small-model “scout” for routing queries [Canonical reading](https://escapemeridian.com/readings/milios-and-pal-propose-a-small-model-scout-for-routing-queries-67e8d16d) In their paper, Aristides Milios and Christopher Pal propose using the prefill activations of a small, frozen language model—the “scout”—to predict both whether candidate models will succeed and how many output tokens they will use. A router can then choose a model per query using estimated success and cost, rather than pricing every query as though it had the same output length. The authors argue that this distinction matters especially for reasoning models, whose output lengths vary widely. In their experiments, predicted costs reduced spending by 9–33% at matched accuracy on eight of nine datasets, compared with pricing based on median output length. The reported savings reached up to 69% of the savings possible with perfect per-query cost knowledge. The abstract says output length tracks problem difficulty on most math and code datasets, but not on knowledge and mixed-format datasets; there, predicting success alone estimated cost less well than a probe trained on length. The authors also report that a length probe retained most of its savings with 20 labelled problems and transferred without labels to four of seven new benchmarks. Savings stopped growing at about a 2B scout; above 4B, the authors say, the larger scout cost more than it added. These are reported benchmark results, not evidence in the abstract about performance in deployed routing systems. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 6, 2026 (2026-11-06). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-12T22:05:52.318Z. First observed: 2026-11-09T05:07:02.544Z. Retrieved: 2026-11-12T22:04:35.738Z. - [Scout Before You Route: Predicting Task Success and Token Count from a Small LLM Prefill](https://arxiv.org/abs/2611.09028) ## Limits The supplied source text gives the abstract but not the paper's detailed experimental setup or the identities of the nine datasets. The abstract reports benchmark savings; it does not report performance in a deployed routing system. ## Open question How would the reported cost savings change when the scout and candidate models route queries in a deployed system? ## Source-stated Milios and Pal propose predicting candidate-model success probability and output length from the prefill activations of one small, frozen LLM. > We predict both the success probability and the output length of candidate models from the prefill activations of a single small frozen LLM, the scout. - [Supporting source](https://arxiv.org/abs/2611.09028) ## Source-stated On eight of nine datasets, the authors report that predicted costs reduced spending by 9–33% at matched accuracy compared with median-length pricing. > On those eight datasets, our predicted costs save 9-33% at matched accuracy relative to median-length pricing - [Supporting source](https://arxiv.org/abs/2611.09028) ## Source-stated The authors report that output length tracks problem difficulty on most math and code datasets, while predicted success estimates cost less effectively on knowledge and mixed-format datasets. > On most math and code datasets, harder problems are longer, so predicted success alone prices cost nearly as well. On knowledge and mixed-format datasets they are not - [Supporting source](https://arxiv.org/abs/2611.09028) ## Source-stated The abstract reports that the length probe retained most of its savings with 20 labelled problems and transferred without labels to four of seven new benchmarks. > The probe keeps most of its saving with 20 labelled problems and transfers without labels to four of seven new benchmarks. - [Supporting source](https://arxiv.org/abs/2611.09028) ## Source-stated The authors report that savings stop growing at about a 2B scout, and that above 4B a larger scout costs more than it adds. > Savings stop growing at about a 2B scout, so above 4B a larger scout costs more than it adds. - [Supporting source](https://arxiv.org/abs/2611.09028) ## Source and contributor context Milios, Aristides, Pal, Christopher · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Model tokens Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens. - [Definition reference](https://developers.google.com/machine-learning/glossary#token) ## Reasoning model A language model trained to work through a problem in written steps before it answers. - [Definition reference](https://en.wikipedia.org/wiki/Reasoning_model) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 61a9a780-5be5-494c-8ae0-5a9db68a6552. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-12T22:05:52.318Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: c388622665dd805d93aa843f5f531eb1bf0bd3d31248ff5b34ad038c9d0f68bb. Source text hash: 15662cde9e60be440ae9d423fbe4934f077cf960df1d78e42e33e7ad0da84f6a. ## Revision and source-check history ### Version 1 — 2026-11-12T22:05:52.318Z In their paper, Aristides Milios and Christopher Pal propose using the prefill activations of a small, frozen language model—the “scout”—to predict both whether candidate models will succeed and how many output tokens they will use. A router can then choose a model per query using estimated success and cost, rather than pricing every query as though it had the same output length. The authors argue that this distinction matters especially for reasoning models, whose output lengths vary widely. In their experiments, predicted costs reduced spending by 9–33% at matched accuracy on eight of nine datasets, compared with pricing based on median output length. The reported savings reached up to 69% of the savings possible with perfect per-query cost knowledge. The abstract says output length tracks problem difficulty on most math and code datasets, but not on knowledge and mixed-format datasets; there, predicting success alone estimated cost less well than a probe trained on length. The authors also report that a length probe retained most of its savings with 20 labelled problems and transferred without labels to four of seven new benchmarks. Savings stopped growing at about a 2B scout; above 4B, the authors say, the larger scout cost more than it added. These are reported benchmark results, not evidence in the abstract about performance in deployed routing systems. ## Original source arXiv cs.AI · Article Original source date: Nov 6, 2026 (2026-11-06). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-12T22:05:52.318Z. First observed: 2026-11-09T05:07:02.544Z. Retrieved: 2026-11-12T22:04:35.738Z. - [Scout Before You Route: Predicting Task Success and Token Count from a Small LLM Prefill](https://arxiv.org/abs/2611.09028) ## Limits The supplied source text gives the abstract but not the paper's detailed experimental setup or the identities of the nine datasets. The abstract reports benchmark savings; it does not report performance in a deployed routing system. ## Open question How would the reported cost savings change when the scout and candidate models route queries in a deployed system? ## Source-stated Milios and Pal propose predicting candidate-model success probability and output length from the prefill activations of one small, frozen LLM. > We predict both the success probability and the output length of candidate models from the prefill activations of a single small frozen LLM, the scout. - [Supporting source](https://arxiv.org/abs/2611.09028) ## Source-stated On eight of nine datasets, the authors report that predicted costs reduced spending by 9–33% at matched accuracy compared with median-length pricing. > On those eight datasets, our predicted costs save 9-33% at matched accuracy relative to median-length pricing - [Supporting source](https://arxiv.org/abs/2611.09028) ## Source-stated The authors report that output length tracks problem difficulty on most math and code datasets, while predicted success estimates cost less effectively on knowledge and mixed-format datasets. > On most math and code datasets, harder problems are longer, so predicted success alone prices cost nearly as well. On knowledge and mixed-format datasets they are not - [Supporting source](https://arxiv.org/abs/2611.09028) ## Source-stated The abstract reports that the length probe retained most of its savings with 20 labelled problems and transferred without labels to four of seven new benchmarks. > The probe keeps most of its saving with 20 labelled problems and transfers without labels to four of seven new benchmarks. - [Supporting source](https://arxiv.org/abs/2611.09028) ## Source-stated The authors report that savings stop growing at about a 2B scout, and that above 4B a larger scout costs more than it adds. > Savings stop growing at about a 2B scout, so above 4B a larger scout costs more than it adds. - [Supporting source](https://arxiv.org/abs/2611.09028) ## Source and contributor context Milios, Aristides, Pal, Christopher · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Model tokens Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens. - [Definition reference](https://developers.google.com/machine-learning/glossary#token) ## Reasoning model A language model trained to work through a problem in written steps before it answers. - [Definition reference](https://en.wikipedia.org/wiki/Reasoning_model) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 61a9a780-5be5-494c-8ae0-5a9db68a6552. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-12T22:05:52.318Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: c388622665dd805d93aa843f5f531eb1bf0bd3d31248ff5b34ad038c9d0f68bb. Source text hash: 15662cde9e60be440ae9d423fbe4934f077cf960df1d78e42e33e7ad0da84f6a. ### passed — 2026-11-12T22:05:52.318Z — anthropic/claude-opus-5.5 Publication: 61a9a780-5be5-494c-8ae0-5a9db68a6552 attribution — pass: The reading correctly attributes the paper and its proposals to Aristides Milios and Christopher Pal, the listed authors. Results are framed as the authors' reported findings, not as established facts. > Authors:Aristides Milios, Christopher Pal summary — pass: The title, summary and question match the abstract: a small frozen scout LLM predicts success and output length, with reported savings, the difficulty–length pattern, label efficiency, transfer and scout-size limits. The router concept is explained in plain terms. Technical terms such as prefill activations, probe and 2B are kept exactly and suit tooltips. Two small repairs: 'less well' softens the source's 'well behind'; 'falls well behind' would restore its force. The 22% mixture result and the stronger-than-baselines comparison are omitted, but this does not distort the reading. > On knowledge and mixed-format datasets they are not, and predicted success falls well behind a probe trained on length. date — pass: The originalPublished date of 2026-11-06 matches the v1 submission date on arXiv. The later feed date of November 9 is not used. > \[Submitted on 6 Nov 2026\] support — pass: Each claim has a verbatim excerpt and is stated as reported by the source. The reading explicitly says these are benchmark results, not evidence about deployed systems, which keeps the gap between test results and real-world use. The question is an open issue that follows from the evidence rather than a leading claim. No benefits or costs are invented. > On those eight datasets, our predicted costs save 9-33% at matched accuracy relative to median-length pricing, up to 69% of what perfect knowledge would save. media — pass: The selected media is the verified direct link to the original arXiv abstract page, which matches the source URL. It is the most direct way to reach the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/67e8d16d38d29a0454ee192ac24072e8ecfe2382ba5ebb1fdbdfb567794cccc5/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/milios-and-pal-propose-a-small-model-scout-for-routing-queries-67e8d16d.json)