Ranjan, Ekagra, Venkitesh, Bharat · Nov 3, 2026
arXiv cs.AI: Hybrid Spark: Adaptive Speculative Decoding That Lets the Content Choose the Drafter
Markdown · JSON · Original source
In an abstract submitted on 3 November 2026, Ekagra Ranjan and Bharat Venkitesh present Hybrid Spark, a framework for speculative decoding that runs a model-based drafter and a model-free drafter on every step. Speculative decoding uses draft tokens that a model then verifies; the authors say its gains depend on both workload shape, such as batch size and , and content. Their approach compares the drafters’ proposed chains using a price per verified draft token: the goodput that slot could earn elsewhere in the batch. They say calibrated acceptance estimates let model-free drafts take part in adaptive verification, which otherwise supports only one model-based drafter. In vLLM, with an n-gram drafter, the authors report 1.10–1.62× the output throughput of their model-based baseline on code editing at batch size 4, and 1.04–1.27× at batch size 128. On chat at temperature 0, they report 0.97–1.01× baseline throughput. They argue this could let one deployment serve both workloads, rather than fixing one drafter or splitting between them. The reported results concern these stated workloads and configurations; the abstract does not provide further evaluation details.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 3, 2026 (2026-11-03).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-09T11:33:56.170Z. First observed: 2026-11-05T05:00:29.575Z. Retrieved: 2026-11-09T11:32:58.076Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The supplied evidence is the page and abstract, not the full paper; it gives no further evaluation details for the reported throughput results. The reported results are for the stated code-editing and chat configurations; the abstract does not establish performance on other workloads.
Open question
How well do the calibrated acceptance estimates, fit once on general text, hold across the code-editing and chat workloads reported?
Source-stated
The authors present a framework that runs both a model-based and a model-free drafter at each step, then gives each request the better of their chains within adaptive verification.
runs a model-based and a model-free drafter on every step and gives each request the better of their chains
Source-stated
The framework uses calibrated acceptance estimates for model-free drafter tokens to compare both drafters’ chains through a price per verified draft token.
Once the model-free drafter's tokens carry calibrated acceptance estimates, fit once on general text, this price puts both drafters' chains on one scale.
Source-stated
With an n-gram drafter in vLLM, the authors report 1.10–1.62× the output throughput of their model-based baseline on code editing at batch size 4.
achieves 1.10--1.62\x the output throughput of \dspark on code editing at batch size~4
Source-stated
With the same setup, the authors report 1.04–1.27× baseline output throughput on code editing at batch size 128.
1.04--1.27\x at batch size~128
Source-stated
On chat at temperature 0, the authors report throughput of 0.97–1.01× the baseline.
while staying at 0.97--1.01\x of it on chat at temperature~0.
Source-stated
The authors say the framework keeps asynchronous scheduling intact and enables one deployment to serve both workloads.
entirely on the GPU, which keeps asynchronous scheduling intact
Source and contributor context
Ranjan, Ekagra, Venkitesh, Bharat · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Context window
How much text a model can take in and consider at one time.
Graphics processing unit
A chip that performs many calculations at once; the main hardware used to train and run AI models.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 3a71566a-dce4-4290-8ec9-0bd0bc0c3962.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T11:33:56.170Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 1ce58eef01b3810bda6817d6c3b1c73364bcfabebf0820fcae351bfe08ed33ed. Source text hash: ea3053afff99fc7f97f195930a717c29477513dc210a3af2f2b7da5f1c40235b.
Revision and source-check history
Version 1 · 2026-11-09T11:33:56.170Z
arXiv cs.AI: Hybrid Spark: Adaptive Speculative Decoding That Lets the Content Choose the Drafter
In an abstract submitted on 3 November 2026, Ekagra Ranjan and Bharat Venkitesh present Hybrid Spark, a framework for speculative decoding that runs a model-based drafter and a model-free drafter on every step. Speculative decoding uses draft tokens that a model then verifies; the authors say its gains depend on both workload shape, such as batch size and , and content. Their approach compares the drafters’ proposed chains using a price per verified draft token: the goodput that slot could earn elsewhere in the batch. They say calibrated acceptance estimates let model-free drafts take part in adaptive verification, which otherwise supports only one model-based drafter. In vLLM, with an n-gram drafter, the authors report 1.10–1.62× the output throughput of their model-based baseline on code editing at batch size 4, and 1.04–1.27× at batch size 128. On chat at temperature 0, they report 0.97–1.01× baseline throughput. They argue this could let one deployment serve both workloads, rather than fixing one drafter or splitting between them. The reported results concern these stated workloads and configurations; the abstract does not provide further evaluation details.
Original source
cs.AI · Article
Source details
Original source date: Nov 3, 2026 (2026-11-03).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-09T11:33:56.170Z. First observed: 2026-11-05T05:00:29.575Z. Retrieved: 2026-11-09T11:32:58.076Z.
Limits
The supplied evidence is the page and abstract, not the full paper; it gives no further evaluation details for the reported throughput results. The reported results are for the stated code-editing and chat configurations; the abstract does not establish performance on other workloads.
Open question
How well do the calibrated acceptance estimates, fit once on general text, hold across the code-editing and chat workloads reported?
Source-stated
The authors present a framework that runs both a model-based and a model-free drafter at each step, then gives each request the better of their chains within adaptive verification.
runs a model-based and a model-free drafter on every step and gives each request the better of their chains
Source-stated
The framework uses calibrated acceptance estimates for model-free drafter tokens to compare both drafters’ chains through a price per verified draft token.
Once the model-free drafter's tokens carry calibrated acceptance estimates, fit once on general text, this price puts both drafters' chains on one scale.
Source-stated
With an n-gram drafter in vLLM, the authors report 1.10–1.62× the output throughput of their model-based baseline on code editing at batch size 4.
achieves 1.10--1.62\x the output throughput of \dspark on code editing at batch size~4
Source-stated
With the same setup, the authors report 1.04–1.27× baseline output throughput on code editing at batch size 128.
1.04--1.27\x at batch size~128
Source-stated
On chat at temperature 0, the authors report throughput of 0.97–1.01× the baseline.
while staying at 0.97--1.01\x of it on chat at temperature~0.
Source-stated
The authors say the framework keeps asynchronous scheduling intact and enables one deployment to serve both workloads.
entirely on the GPU, which keeps asynchronous scheduling intact
Source and contributor context
Ranjan, Ekagra, Venkitesh, Bharat · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Context window
How much text a model can take in and consider at one time.
Graphics processing unit
A chip that performs many calculations at once; the main hardware used to train and run AI models.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 3a71566a-dce4-4290-8ec9-0bd0bc0c3962.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T11:33:56.170Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 1ce58eef01b3810bda6817d6c3b1c73364bcfabebf0820fcae351bfe08ed33ed. Source text hash: ea3053afff99fc7f97f195930a717c29477513dc210a3af2f2b7da5f1c40235b.
uncertain · 2026-11-09T11:33:56.170Z · anthropic/claude-opus-5.5
Publication: 3a71566a-dce4-4290-8ec9-0bd0bc0c3962
attribution — pass: The draft correctly attributes Hybrid Spark to Ekagra Ranjan and Bharat Venkitesh and keeps the throughput results as author-reported. The abstract names the baseline only through an unrendered macro, so 'their model-based baseline' slightly implies the authors built it. 'the model-based baseline used in the paper' would be safer, but this does not misattribute the claims.
Authors:Ekagra Ranjan, Bharat Venkitesh
summary — uncertain: The reading is accurate but omits the abstract's central mechanism: a model-free drafter such as n-gram lookup copies long runs cheaply when the output repeats its input, as in code editing. That is why the content matters and why code editing gains. 'Model-free drafter', 'goodput', 'batch size' and 'temperature 0' are not explained, and speculative decoding is described only as 'draft tokens that a model then verifies'. 'could let one deployment' also softens the abstract's direct 'can therefore'. Smallest repair: add the n-gram copying explanation from the source and suggest tooltips for these terms.
a model-free drafter such as n-gram lookup proposes long copies at almost no cost when the output repeats its input, as in code editing
date — pass: The original publication date of 2026-11-03 matches the v1 submission date. It is not the later feed date of 2026-11-05.
[Submitted on 3 Nov 2026]
support — pass: The abstract supports every claim, including the price mechanism, the throughput ranges, batch sizes, chat at temperature 0, asynchronous scheduling and the single-deployment conclusion. The question about calibration fit once on general text follows directly from the source. The limitations correctly note that only the abstract was supplied.
The key idea is a price per verified draft token: the goodput that slot would earn elsewhere in the batch.
media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivHow much text a model can take in and consider at one time.
Definition referenceA chip that performs many calculations at once; the main hardware used to train and run AI models.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivHow much text a model can take in and consider at one time.
Definition referenceA chip that performs many calculations at once; the main hardware used to train and run AI models.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv