Wang, Haiyu, Zhang, Sai Qian, Liu, Zechun, Li, Steven · Nov 2, 2026
Haiyu Wang and coauthors propose HybridSpec to speed speculative decoding
Markdown · JSON · Original source
Haiyu Wang and coauthors present HybridSpec, a method for choosing how many draft tokens a target model checks in each round of speculative decoding. A draft proposes tokens in parallel; the target verifies them. Checking more tokens can accept more at once, but work after a rejection is wasted. The authors say this tradeoff is especially costly for hybrid models combining Gated DeltaNet (GDN) and (MoE) layers: verification rose 71% between lengths 4 and 16 in their Qwen3.6-35B-A3B profile, compared with 4%-6% for a dense-model comparison. HybridSpec chooses the verification length using draft-side acceptance estimates and measured target . It also checks routing information before running experts, then uses observed expert assignments to decide whether to keep a token. The authors report that, in single-request greedy decoding with three GDN-MoE target models, HybridSpec was 1.75×-1.88× faster than autoregressive decoding and 1.30×-1.33× faster than DFlash. These results suggest the method may reduce decoding time in the tested setting; the abstract does not establish how it performs across broader workloads or serving conditions. How does HybridSpec’s speedup change across workloads with different request sizes and draft-token acceptance rates?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 2, 2026 (2026-11-02).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T05:16:49.959Z. First observed: 2026-11-04T05:15:26.455Z. Retrieved: 2026-11-04T06:32:26.448Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The abstract reports single-request results; it does not specify performance across concurrent requests or broader serving conditions. The isolated block-parallel GDN kernel comparison assumes every draft token is accepted, so it does not by itself establish kernel performance under rejection. The evidence is the paper’s abstract, which does not provide further methodology or results across varied request sizes and draft-token acceptance rates.
Open question
How does HybridSpec’s speedup change across workloads with different request sizes and draft-token acceptance rates?
Source-stated
The authors say verification time in their Qwen3.6-35B-A3B single-request profile grew 71% from verification length 4 to 16, compared with 4%-6% for a dense-model comparison.
In a single-request profile of Qwen3.6-35B-A3B with DFlash, it grew by 71% from length 4 to 16, almost entirely in GDN and MoE layers, compared with 4%-6% for a dense Qwen3-4B target.
Source-stated
HybridSpec selects verification length using draft-side acceptance estimates and measured target latency, and uses observed expert assignments to decide whether to keep a token.
selects the verification length on the GPU from draft-side acceptance estimates and measured target latency. Verification then runs this length plus one extra token up to the first MoE router, before any expert runs. There, a lightweight routing correction uses the observed expert assignments to dec
Source-stated
In single-request greedy decoding with three GDN-MoE target models, the authors report HybridSpec was 1.75×-1.88× faster than autoregressive decoding and 1.30×-1.33× faster than DFlash.
With three GDN-MoE target models in single-request greedy decoding, HybridSpec is $1.75\times$-$1.88\times$ faster than autoregressive decoding and $1.30\times$-$1.33\times$ faster than DFlash.
Escape Meridian interpretation
The reported results indicate speedups in the specified tests, but do not establish performance across broader workloads or serving conditions.
With three GDN-MoE target models in single-request greedy decoding, HybridSpec is $1.75\times$-$1.88\times$ faster than autoregressive decoding and $1.30\times$-$1.33\times$ faster than DFlash.
Source and contributor context
Wang, Haiyu, Zhang, Sai Qian, Liu, Zechun, Li, Steven · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Mixture of experts
A model design that sends each input to a few specialized sub-networks instead of the whole model, which saves computation.
Latency
The delay between sending a request and receiving the result.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: 98f8c2bd-7509-4137-8af5-aa0d3027a4b9.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T05:16:49.959Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 9a5fa7c6e6142091e3558d8e031a183a161d92ec168cf61a56812c18fbbdf463. Source text hash: 978748444196c7ed0ad6711271179d812f0693ef1a6a7a2093a8ce3de5fa99c3.
Revision and source-check history
Version 1 · 2026-11-10T05:16:49.959Z
Haiyu Wang and coauthors propose HybridSpec to speed speculative decoding
Haiyu Wang and coauthors present HybridSpec, a method for choosing how many draft tokens a target model checks in each round of speculative decoding. A draft proposes tokens in parallel; the target verifies them. Checking more tokens can accept more at once, but work after a rejection is wasted. The authors say this tradeoff is especially costly for hybrid models combining Gated DeltaNet (GDN) and (MoE) layers: verification rose 71% between lengths 4 and 16 in their Qwen3.6-35B-A3B profile, compared with 4%-6% for a dense-model comparison. HybridSpec chooses the verification length using draft-side acceptance estimates and measured target . It also checks routing information before running experts, then uses observed expert assignments to decide whether to keep a token. The authors report that, in single-request greedy decoding with three GDN-MoE target models, HybridSpec was 1.75×-1.88× faster than autoregressive decoding and 1.30×-1.33× faster than DFlash. These results suggest the method may reduce decoding time in the tested setting; the abstract does not establish how it performs across broader workloads or serving conditions. How does HybridSpec’s speedup change across workloads with different request sizes and draft-token acceptance rates?
Original source
cs.AI · Article
Source details
Original source date: Nov 2, 2026 (2026-11-02).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T05:16:49.959Z. First observed: 2026-11-04T05:15:26.455Z. Retrieved: 2026-11-04T06:32:26.448Z.
Limits
The abstract reports single-request results; it does not specify performance across concurrent requests or broader serving conditions. The isolated block-parallel GDN kernel comparison assumes every draft token is accepted, so it does not by itself establish kernel performance under rejection. The evidence is the paper’s abstract, which does not provide further methodology or results across varied request sizes and draft-token acceptance rates.
Open question
How does HybridSpec’s speedup change across workloads with different request sizes and draft-token acceptance rates?
Source-stated
The authors say verification time in their Qwen3.6-35B-A3B single-request profile grew 71% from verification length 4 to 16, compared with 4%-6% for a dense-model comparison.
In a single-request profile of Qwen3.6-35B-A3B with DFlash, it grew by 71% from length 4 to 16, almost entirely in GDN and MoE layers, compared with 4%-6% for a dense Qwen3-4B target.
Source-stated
HybridSpec selects verification length using draft-side acceptance estimates and measured target latency, and uses observed expert assignments to decide whether to keep a token.
selects the verification length on the GPU from draft-side acceptance estimates and measured target latency. Verification then runs this length plus one extra token up to the first MoE router, before any expert runs. There, a lightweight routing correction uses the observed expert assignments to dec
Source-stated
In single-request greedy decoding with three GDN-MoE target models, the authors report HybridSpec was 1.75×-1.88× faster than autoregressive decoding and 1.30×-1.33× faster than DFlash.
With three GDN-MoE target models in single-request greedy decoding, HybridSpec is $1.75\times$-$1.88\times$ faster than autoregressive decoding and $1.30\times$-$1.33\times$ faster than DFlash.
Escape Meridian interpretation
The reported results indicate speedups in the specified tests, but do not establish performance across broader workloads or serving conditions.
With three GDN-MoE target models in single-request greedy decoding, HybridSpec is $1.75\times$-$1.88\times$ faster than autoregressive decoding and $1.30\times$-$1.33\times$ faster than DFlash.
Source and contributor context
Wang, Haiyu, Zhang, Sai Qian, Liu, Zechun, Li, Steven · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Mixture of experts
A model design that sends each input to a few specialized sub-networks instead of the whole model, which saves computation.
Latency
The delay between sending a request and receiving the result.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: 98f8c2bd-7509-4137-8af5-aa0d3027a4b9.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T05:16:49.959Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 9a5fa7c6e6142091e3558d8e031a183a161d92ec168cf61a56812c18fbbdf463. Source text hash: 978748444196c7ed0ad6711271179d812f0693ef1a6a7a2093a8ce3de5fa99c3.
uncertain · 2026-11-10T05:16:49.959Z · anthropic/claude-opus-5.5
Publication: 98f8c2bd-7509-4137-8af5-aa0d3027a4b9
attribution — pass: The draft correctly attributes the HybridSpec work and its reported measurements to Haiyu Wang and coauthors, the arXiv authors. It frames the results as the authors' claims rather than established fact.
Authors:Haiyu Wang, Sai Qian Zhang, Zechun Liu, Steven Li
summary — uncertain: The draft is broadly faithful but may not be clear to a general reader. 'Autoregressive decoding', 'greedy decoding', 'DFlash', 'experts' and 'router' are central but unexplained. DFlash is the baseline, and the draft does not say it is a parallel drafter. The draft also drops the source's explanation of why hybrid verification is costly: GDN layers update and recover a recurrent state, and MoE cost depends on which experts the tokens activate. It omits the block-parallel GDN kernel contribution as well. Smallest repair: add one plain-language line on each mechanism and identify DFlash as the parallel drafter baseline.
Each GDN layer must update a recurrent state at every token and recover it at the end of the accepted prefix. The MoE cost depends on the experts that the tokens activate, which are known only once verification runs.
date — pass: The originalPublished date of 2026-11-02 matches the arXiv v1 submission date. This is the first publication, not the later feed entry.
[Submitted on 2 Nov 2026]
support — pass: The 71% versus 4%-6% profile, the method description and the 1.75x-1.88x and 1.30x-1.33x speedups are all source-stated. The interpretation is appropriately conditional. The limitations accurately note the single-request scope and the all-accepted kernel benchmark. The question follows from the evidence, though 'request sizes' loosely stands in for concurrency or batching.
In isolated benchmarks at verification lengths 4 to 16 with every draft token accepted
media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A model design that sends each input to a few specialized sub-networks instead of the whole model, which saves computation.
Definition referenceTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEThe delay between sending a request and receiving the result.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA model design that sends each input to a few specialized sub-networks instead of the whole model, which saves computation.
Definition referenceTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEThe delay between sending a request and receiving the result.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference