Milios, Aristides, Pal, Christopher · Nov 6, 2026
Milios and Pal propose a small-model “scout” for routing queries
Markdown · JSON · Original source
In their paper, Aristides Milios and Christopher Pal propose using the prefill activations of a small, frozen language model—the “scout”—to predict both whether candidate models will succeed and how many they will use. A router can then choose a model per query using estimated success and cost, rather than pricing every query as though it had the same output length. The authors argue that this distinction matters especially for , whose output lengths vary widely. In their experiments, predicted costs reduced spending by 9–33% at matched accuracy on eight of nine datasets, compared with pricing based on median output length. The reported savings reached up to 69% of the savings possible with perfect per-query cost knowledge. The abstract says output length tracks problem difficulty on most math and code datasets, but not on knowledge and mixed-format datasets; there, predicting success alone estimated cost less well than a probe trained on length. The authors also report that a length probe retained most of its savings with 20 labelled problems and transferred without labels to four of seven new . Savings stopped growing at about a 2B scout; above 4B, the authors say, the larger scout cost more than it added. These are reported benchmark results, not evidence in the abstract about performance in deployed routing systems.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 6, 2026 (2026-11-06).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-12T22:05:52.318Z. First observed: 2026-11-09T05:07:02.544Z. Retrieved: 2026-11-12T22:04:35.738Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The supplied source text gives the abstract but not the paper's detailed experimental setup or the identities of the nine datasets. The abstract reports savings; it does not report performance in a deployed routing system.
Open question
How would the reported cost savings change when the scout and candidate models route queries in a deployed system?
Source-stated
Milios and Pal propose predicting candidate-model success probability and output length from the prefill activations of one small, frozen LLM.
We predict both the success probability and the output length of candidate models from the prefill activations of a single small frozen LLM, the scout.
Source-stated
On eight of nine datasets, the authors report that predicted costs reduced spending by 9–33% at matched accuracy compared with median-length pricing.
On those eight datasets, our predicted costs save 9-33% at matched accuracy relative to median-length pricing
Source-stated
The authors report that output length tracks problem difficulty on most math and code datasets, while predicted success estimates cost less effectively on knowledge and mixed-format datasets.
On most math and code datasets, harder problems are longer, so predicted success alone prices cost nearly as well. On knowledge and mixed-format datasets they are not
Source-stated
The abstract reports that the length probe retained most of its savings with 20 labelled problems and transferred without labels to four of seven new benchmarks.
The probe keeps most of its saving with 20 labelled problems and transfers without labels to four of seven new benchmarks.
Source-stated
The authors report that savings stop growing at about a 2B scout, and that above 4B a larger scout costs more than it adds.
Savings stop growing at about a 2B scout, so above 4B a larger scout costs more than it adds.
Source and contributor context
Milios, Aristides, Pal, Christopher · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Model tokens
Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens.
Reasoning model
A language model trained to work through a problem in written steps before it answers.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 61a9a780-5be5-494c-8ae0-5a9db68a6552.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-12T22:05:52.318Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: c388622665dd805d93aa843f5f531eb1bf0bd3d31248ff5b34ad038c9d0f68bb. Source text hash: 15662cde9e60be440ae9d423fbe4934f077cf960df1d78e42e33e7ad0da84f6a.
Revision and source-check history
Version 1 · 2026-11-12T22:05:52.318Z
Milios and Pal propose a small-model “scout” for routing queries
In their paper, Aristides Milios and Christopher Pal propose using the prefill activations of a small, frozen language model—the “scout”—to predict both whether candidate models will succeed and how many they will use. A router can then choose a model per query using estimated success and cost, rather than pricing every query as though it had the same output length. The authors argue that this distinction matters especially for , whose output lengths vary widely. In their experiments, predicted costs reduced spending by 9–33% at matched accuracy on eight of nine datasets, compared with pricing based on median output length. The reported savings reached up to 69% of the savings possible with perfect per-query cost knowledge. The abstract says output length tracks problem difficulty on most math and code datasets, but not on knowledge and mixed-format datasets; there, predicting success alone estimated cost less well than a probe trained on length. The authors also report that a length probe retained most of its savings with 20 labelled problems and transferred without labels to four of seven new . Savings stopped growing at about a 2B scout; above 4B, the authors say, the larger scout cost more than it added. These are reported benchmark results, not evidence in the abstract about performance in deployed routing systems.
Original source
cs.AI · Article
Source details
Original source date: Nov 6, 2026 (2026-11-06).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-12T22:05:52.318Z. First observed: 2026-11-09T05:07:02.544Z. Retrieved: 2026-11-12T22:04:35.738Z.
Limits
The supplied source text gives the abstract but not the paper's detailed experimental setup or the identities of the nine datasets. The abstract reports savings; it does not report performance in a deployed routing system.
Open question
How would the reported cost savings change when the scout and candidate models route queries in a deployed system?
Source-stated
Milios and Pal propose predicting candidate-model success probability and output length from the prefill activations of one small, frozen LLM.
We predict both the success probability and the output length of candidate models from the prefill activations of a single small frozen LLM, the scout.
Source-stated
On eight of nine datasets, the authors report that predicted costs reduced spending by 9–33% at matched accuracy compared with median-length pricing.
On those eight datasets, our predicted costs save 9-33% at matched accuracy relative to median-length pricing
Source-stated
The authors report that output length tracks problem difficulty on most math and code datasets, while predicted success estimates cost less effectively on knowledge and mixed-format datasets.
On most math and code datasets, harder problems are longer, so predicted success alone prices cost nearly as well. On knowledge and mixed-format datasets they are not
Source-stated
The abstract reports that the length probe retained most of its savings with 20 labelled problems and transferred without labels to four of seven new benchmarks.
The probe keeps most of its saving with 20 labelled problems and transfers without labels to four of seven new benchmarks.
Source-stated
The authors report that savings stop growing at about a 2B scout, and that above 4B a larger scout costs more than it adds.
Savings stop growing at about a 2B scout, so above 4B a larger scout costs more than it adds.
Source and contributor context
Milios, Aristides, Pal, Christopher · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Model tokens
Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens.
Reasoning model
A language model trained to work through a problem in written steps before it answers.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 61a9a780-5be5-494c-8ae0-5a9db68a6552.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-12T22:05:52.318Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: c388622665dd805d93aa843f5f531eb1bf0bd3d31248ff5b34ad038c9d0f68bb. Source text hash: 15662cde9e60be440ae9d423fbe4934f077cf960df1d78e42e33e7ad0da84f6a.
passed · 2026-11-12T22:05:52.318Z · anthropic/claude-opus-5.5
Publication: 61a9a780-5be5-494c-8ae0-5a9db68a6552
attribution — pass: The reading correctly attributes the paper and its proposals to Aristides Milios and Christopher Pal, the listed authors. Results are framed as the authors' reported findings, not as established facts.
Authors:Aristides Milios, Christopher Pal
summary — pass: The title, summary and question match the abstract: a small frozen scout LLM predicts success and output length, with reported savings, the difficulty–length pattern, label efficiency, transfer and scout-size limits. The router concept is explained in plain terms. Technical terms such as prefill activations, probe and 2B are kept exactly and suit tooltips. Two small repairs: 'less well' softens the source's 'well behind'; 'falls well behind' would restore its force. The 22% mixture result and the stronger-than-baselines comparison are omitted, but this does not distort the reading.
On knowledge and mixed-format datasets they are not, and predicted success falls well behind a probe trained on length.
date — pass: The originalPublished date of 2026-11-06 matches the v1 submission date on arXiv. The later feed date of November 9 is not used.
[Submitted on 6 Nov 2026]
support — pass: Each claim has a verbatim excerpt and is stated as reported by the source. The reading explicitly says these are benchmark results, not evidence about deployed systems, which keeps the gap between test results and real-world use. The question is an open issue that follows from the evidence rather than a leading claim. No benefits or costs are invented.
On those eight datasets, our predicted costs save 9-33% at matched accuracy relative to median-length pricing, up to 69% of what perfect knowledge would save.
media — pass: The selected media is the verified direct link to the original arXiv abstract page, which matches the source URL. It is the most direct way to reach the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens.
Definition referenceA language model trained to work through a problem in written steps before it answers.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceSmall chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens.
Definition referenceA language model trained to work through a problem in written steps before it answers.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference