{"id":"c0aa900313c4fdaeb65295f95584f5e7cf2553b53fb5ff0a51f6df175d1eecd7","title":"Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models","argument":"Authors Ekagra Ranjan, Komal K. Teru, Bharat Venkitesh and Acyr Locatelli argue that speculative decoding can speed up sparse MoE models more at low batch sizes than prior predictions suggest. Speculative decoding drafts several tokens and checks them in one pass; MoE models route work to selected expert networks. The authors evaluate 10 models and report that recent fine-grained models show higher speedup at low batch size, contrary to the predicted pattern of gains peaking at intermediate batch sizes. They attribute the difference to verification costs: repeated reads can be absorbed by L2, GPU bandwidth use differs between MoE layers and attention, and draft tokens may route to overlapping experts. Their model uses routing-overlap statistics to predict how many unique experts verification activates. The authors report less than 4% error and say it narrows the gap to measured model curves. This offers a possible way to better estimate inference speedups, but the abstract does not establish how the findings translate to other hardware or deployments. Do these results hold beyond the evaluated models and H100 profiling?","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The available evidence is the paper's abstract; it does not provide the experimental setup, detailed results, or the full basis for comparing predicted and observed speedups. The reported kernel profiling is specifically described for H100; the abstract does not establish whether the findings generalize to other hardware or deployments. The abstract reports evaluation across sparse MoE models but does not identify their names or give model-by-model results.","speaker":{"name":"Ranjan, Ekagra, Teru, Komal K., Venkitesh, Bharat, Locatelli, Acyr","roleAtSource":"Article author"},"source":{"title":"Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.38981","publishedAt":"2026-10-28","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors report evaluating 10 sparse MoE models and finding recent fine-grained models show higher speculative-decoding speedup at low batch sizes, contrary to prior predictions.","excerpt":"We evaluate this across 10 sparse MoE models (30B--1T parameters, 8--512 experts) and find recent fine-grained models break this trend, showing higher speedup at low batch size.","url":"https://arxiv.org/abs/2610.38981","relationship":"Source-stated"},{"text":"The authors say verification costs depend on more than the bytes of activated experts, including draft-token routing overlap and GPU execution dynamics.","excerpt":"We find verification cost is governed not only by activated-expert bytes, but also by temporal overlap across draft tokens and GPU execution dynamics.","url":"https://arxiv.org/abs/2610.38981","relationship":"Source-stated"},{"text":"For low batch sizes, the authors attribute a smaller-than-expected runtime share for MoE layers partly to GPU bandwidth use and L2 cache behavior on H100.","excerpt":"on H100, kernel profiling attributes this mainly to higher bandwidth utilization of MoE kernels relative to attention, with L2 absorbing repeated reads of co-routed experts.","url":"https://arxiv.org/abs/2610.38981","relationship":"Source-stated"},{"text":"The authors report that their Iterative Proportional Fitting framework predicts unique expert counts with less than 4% error.","excerpt":"We model this effect using a maximum-entropy Iterative Proportional Fitting (IPF) framework that predicts unique expert counts from routing-overlap statistics with less than 4\\% error","url":"https://arxiv.org/abs/2610.38981","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.38981","checkedAt":"2026-11-10T19:46:10.718Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Do these results hold beyond the evaluated models and H100 profiling?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models","summary":"Authors Ekagra Ranjan, Komal K. Teru, Bharat Venkitesh and Acyr Locatelli argue that speculative decoding can speed up sparse MoE models more at low batch sizes than prior predictions suggest. Speculative decoding drafts several tokens and checks them in one pass; MoE models route work to selected expert networks. The authors evaluate 10 models and report that recent fine-grained models show higher speedup at low batch size, contrary to the predicted pattern of gains peaking at intermediate batch sizes. They attribute the difference to verification costs: repeated reads can be absorbed by L2, GPU bandwidth use differs between MoE layers and attention, and draft tokens may route to overlapping experts. Their model uses routing-overlap statistics to predict how many unique experts verification activates. The authors report less than 4% error and say it narrows the gap to measured model curves. This offers a possible way to better estimate inference speedups, but the abstract does not establish how the findings translate to other hardware or deployments. Do these results hold beyond the evaluated models and H100 profiling?","question":"Do these results hold beyond the evaluated models and H100 profiling?","claims":[{"text":"The authors report evaluating 10 sparse MoE models and finding recent fine-grained models show higher speculative-decoding speedup at low batch sizes, contrary to prior predictions.","excerpt":"We evaluate this across 10 sparse MoE models (30B--1T parameters, 8--512 experts) and find recent fine-grained models break this trend, showing higher speedup at low batch size.","support":"source-stated"},{"text":"The authors say verification costs depend on more than the bytes of activated experts, including draft-token routing overlap and GPU execution dynamics.","excerpt":"We find verification cost is governed not only by activated-expert bytes, but also by temporal overlap across draft tokens and GPU execution dynamics.","support":"source-stated"},{"text":"For low batch sizes, the authors attribute a smaller-than-expected runtime share for MoE layers partly to GPU bandwidth use and L2 cache behavior on H100.","excerpt":"on H100, kernel profiling attributes this mainly to higher bandwidth utilization of MoE kernels relative to attention, with L2 absorbing repeated reads of co-routed experts.","support":"source-stated"},{"text":"The authors report that their Iterative Proportional Fitting framework predicts unique expert counts with less than 4% error.","excerpt":"We model this effect using a maximum-entropy Iterative Proportional Fitting (IPF) framework that predicts unique expert counts from routing-overlap statistics with less than 4\\% error","support":"source-stated"}],"limitations":["The available evidence is the paper's abstract; it does not provide the experimental setup, detailed results, or the full basis for comparing predicted and observed speedups.","The reported kernel profiling is specifically described for H100; the abstract does not establish whether the findings generalize to other hardware or deployments.","The abstract reports evaluation across sparse MoE models but does not identify their names or give model-by-model results."],"mediaId":"52a229047e6f88ced36d02cd7d7436dcba50185e6acbdbd061c75d6a437aeca8","correctionReason":null,"id":"c0aa900313c4fdaeb65295f95584f5e7cf2553b53fb5ff0a51f6df175d1eecd7","publicationId":"44732b85-407f-426d-8717-832e13ad9bf2","version":1,"sourceUrl":"https://arxiv.org/abs/2610.38981","sourceTitle":"Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-28","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-30T06:45:33.877Z","createdAt":"2026-11-10T19:31:58.625Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"52a229047e6f88ced36d02cd7d7436dcba50185e6acbdbd061c75d6a437aeca8","kind":"article","url":"https://arxiv.org/abs/2610.38981","sourceUrl":"https://arxiv.org/abs/2610.38981","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-30T16:32:28.253Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.38981","checkedAt":"2026-11-10T19:46:10.718Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"52a229047e6f88ced36d02cd7d7436dcba50185e6acbdbd061c75d6a437aeca8","confidence":0.8500000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"d4fdbdd29e3c3d4a9870e10a3147acd3097aeda7ae0855230be3b55d9ee2f1a2","decidedAt":"2026-10-30T16:32:32.108Z"},"provenance":{"textHash":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","observationId":"d5ba7a50062210c739c073a95b52dae42149782d289fe246a2bf093bc7b08ed2","retrievedAt":"2026-10-30T16:32:27.669Z","kind":"publisher-page","preparer":{"model":"server-excerpt-repair/stream-excerpt-repair-v1 (original draft: gpt-6-luna)","runId":"3650bd46-3e38-4be9-8dcd-a8b575f8f13e"},"reviewer":{"model":"gpt-6-luna","runId":"704d3534-e066-45be-8d03-9fe21f57b545"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T19:31:58.625Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Ranjan, Ekagra","url":null,"kind":"unknown"},{"name":"Teru, Komal K.","url":null,"kind":"unknown"},{"name":"Venkitesh, Bharat","url":null,"kind":"unknown"},{"name":"Locatelli, Acyr","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Ranjan, Ekagra","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ranjan, Ekagra","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38981","locator":"head/meta[name=author|citation_author]"},"assertionId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T19:46:10.718Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426","sourceUrl":"https://arxiv.org/abs/2610.38981","contentSha256":"cd3bc13cef0c5b665d1dcf3f7eb779bdc7aa769a775907476a3cecf947a60044","textHash":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","observedAt":"2026-11-10T19:46:10.718Z"}},{"actor":{"name":"Teru, Komal K.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Teru, Komal K.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38981","locator":"head/meta[name=author|citation_author]"},"assertionId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T19:46:10.718Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426","sourceUrl":"https://arxiv.org/abs/2610.38981","contentSha256":"cd3bc13cef0c5b665d1dcf3f7eb779bdc7aa769a775907476a3cecf947a60044","textHash":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","observedAt":"2026-11-10T19:46:10.718Z"}},{"actor":{"name":"Venkitesh, Bharat","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Venkitesh, Bharat","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38981","locator":"head/meta[name=author|citation_author]"},"assertionId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T19:46:10.718Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426","sourceUrl":"https://arxiv.org/abs/2610.38981","contentSha256":"cd3bc13cef0c5b665d1dcf3f7eb779bdc7aa769a775907476a3cecf947a60044","textHash":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","observedAt":"2026-11-10T19:46:10.718Z"}},{"actor":{"name":"Locatelli, Acyr","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Locatelli, Acyr","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38981","locator":"head/meta[name=author|citation_author]"},"assertionId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T19:46:10.718Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426","sourceUrl":"https://arxiv.org/abs/2610.38981","contentSha256":"cd3bc13cef0c5b665d1dcf3f7eb779bdc7aa769a775907476a3cecf947a60044","textHash":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","observedAt":"2026-11-10T19:46:10.718Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.38981","snapshotId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426","contentSha256":"cd3bc13cef0c5b665d1dcf3f7eb779bdc7aa769a775907476a3cecf947a60044","textHash":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","observedAt":"2026-11-10T19:46:10.718Z","basis":"publisher-page-metadata","sourceLinks":44,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/c0aa900313c4fdaeb65295f95584f5e7cf2553b53fb5ff0a51f6df175d1eecd7/source"}},"canonicalUrl":"https://escapemeridian.com/readings/why-speculative-decoding-works-better-than-predicted-on-sparse-c0aa9003","formats":{"html":"/readings/why-speculative-decoding-works-better-than-predicted-on-sparse-c0aa9003","markdown":"/readings/why-speculative-decoding-works-better-than-predicted-on-sparse-c0aa9003.md","json":"/readings/why-speculative-decoding-works-better-than-predicted-on-sparse-c0aa9003.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-mixture-of-experts","name":"Mixture of experts","explanation":"A model design that sends each input to a few specialized sub-networks instead of the whole model, which saves computation.","referenceUrl":"https://developers.google.com/machine-learning/glossary#mixture-of-experts"},{"id":"term-gpu","name":"Graphics processing unit","explanation":"A chip that performs many calculations at once; the main hardware used to train and run AI models.","referenceUrl":"https://en.wikipedia.org/wiki/Graphics_processing_unit"},{"id":"term-nvidia-h100","name":"NVIDIA H100","explanation":"An NVIDIA graphics processing unit (GPU) built on the company's Hopper architecture and designed to accelerate AI workloads, including large language models.","referenceUrl":"https://www.nvidia.com/en-us/data-center/h100/"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"c0aa900313c4fdaeb65295f95584f5e7cf2553b53fb5ff0a51f6df175d1eecd7","revision":"44732b85-407f-426d-8717-832e13ad9bf2","revisionKind":"publication-id","title":"Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models","url":"https://escapemeridian.com/readings/why-speculative-decoding-works-better-than-predicted-on-sparse-c0aa9003"},"source":{"type":"source","id":"https://arxiv.org/abs/2610.38981","revision":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","revisionKind":"sha256-source-text","title":"Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models","url":"https://arxiv.org/abs/2610.38981"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"c0aa900313c4fdaeb65295f95584f5e7cf2553b53fb5ff0a51f6df175d1eecd7","title":"Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models","argument":"Authors Ekagra Ranjan, Komal K. Teru, Bharat Venkitesh and Acyr Locatelli argue that speculative decoding can speed up sparse MoE models more at low batch sizes than prior predictions suggest. Speculative decoding drafts several tokens and checks them in one pass; MoE models route work to selected expert networks. The authors evaluate 10 models and report that recent fine-grained models show higher speedup at low batch size, contrary to the predicted pattern of gains peaking at intermediate batch sizes. They attribute the difference to verification costs: repeated reads can be absorbed by L2, GPU bandwidth use differs between MoE layers and attention, and draft tokens may route to overlapping experts. Their model uses routing-overlap statistics to predict how many unique experts verification activates. The authors report less than 4% error and say it narrows the gap to measured model curves. This offers a possible way to better estimate inference speedups, but the abstract does not establish how the findings translate to other hardware or deployments. Do these results hold beyond the evaluated models and H100 profiling?","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The available evidence is the paper's abstract; it does not provide the experimental setup, detailed results, or the full basis for comparing predicted and observed speedups. The reported kernel profiling is specifically described for H100; the abstract does not establish whether the findings generalize to other hardware or deployments. The abstract reports evaluation across sparse MoE models but does not identify their names or give model-by-model results.","speaker":{"name":"Ranjan, Ekagra, Teru, Komal K., Venkitesh, Bharat, Locatelli, Acyr","roleAtSource":"Article author"},"source":{"title":"Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.38981","publishedAt":"2026-10-28","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors report evaluating 10 sparse MoE models and finding recent fine-grained models show higher speculative-decoding speedup at low batch sizes, contrary to prior predictions.","excerpt":"We evaluate this across 10 sparse MoE models (30B--1T parameters, 8--512 experts) and find recent fine-grained models break this trend, showing higher speedup at low batch size.","url":"https://arxiv.org/abs/2610.38981","relationship":"Source-stated"},{"text":"The authors say verification costs depend on more than the bytes of activated experts, including draft-token routing overlap and GPU execution dynamics.","excerpt":"We find verification cost is governed not only by activated-expert bytes, but also by temporal overlap across draft tokens and GPU execution dynamics.","url":"https://arxiv.org/abs/2610.38981","relationship":"Source-stated"},{"text":"For low batch sizes, the authors attribute a smaller-than-expected runtime share for MoE layers partly to GPU bandwidth use and L2 cache behavior on H100.","excerpt":"on H100, kernel profiling attributes this mainly to higher bandwidth utilization of MoE kernels relative to attention, with L2 absorbing repeated reads of co-routed experts.","url":"https://arxiv.org/abs/2610.38981","relationship":"Source-stated"},{"text":"The authors report that their Iterative Proportional Fitting framework predicts unique expert counts with less than 4% error.","excerpt":"We model this effect using a maximum-entropy Iterative Proportional Fitting (IPF) framework that predicts unique expert counts from routing-overlap statistics with less than 4\\% error","url":"https://arxiv.org/abs/2610.38981","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.38981","checkedAt":"2026-11-10T19:46:10.718Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Do these results hold beyond the evaluated models and H100 profiling?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models","summary":"Authors Ekagra Ranjan, Komal K. Teru, Bharat Venkitesh and Acyr Locatelli argue that speculative decoding can speed up sparse MoE models more at low batch sizes than prior predictions suggest. Speculative decoding drafts several tokens and checks them in one pass; MoE models route work to selected expert networks. The authors evaluate 10 models and report that recent fine-grained models show higher speedup at low batch size, contrary to the predicted pattern of gains peaking at intermediate batch sizes. They attribute the difference to verification costs: repeated reads can be absorbed by L2, GPU bandwidth use differs between MoE layers and attention, and draft tokens may route to overlapping experts. Their model uses routing-overlap statistics to predict how many unique experts verification activates. The authors report less than 4% error and say it narrows the gap to measured model curves. This offers a possible way to better estimate inference speedups, but the abstract does not establish how the findings translate to other hardware or deployments. Do these results hold beyond the evaluated models and H100 profiling?","question":"Do these results hold beyond the evaluated models and H100 profiling?","claims":[{"text":"The authors report evaluating 10 sparse MoE models and finding recent fine-grained models show higher speculative-decoding speedup at low batch sizes, contrary to prior predictions.","excerpt":"We evaluate this across 10 sparse MoE models (30B--1T parameters, 8--512 experts) and find recent fine-grained models break this trend, showing higher speedup at low batch size.","support":"source-stated"},{"text":"The authors say verification costs depend on more than the bytes of activated experts, including draft-token routing overlap and GPU execution dynamics.","excerpt":"We find verification cost is governed not only by activated-expert bytes, but also by temporal overlap across draft tokens and GPU execution dynamics.","support":"source-stated"},{"text":"For low batch sizes, the authors attribute a smaller-than-expected runtime share for MoE layers partly to GPU bandwidth use and L2 cache behavior on H100.","excerpt":"on H100, kernel profiling attributes this mainly to higher bandwidth utilization of MoE kernels relative to attention, with L2 absorbing repeated reads of co-routed experts.","support":"source-stated"},{"text":"The authors report that their Iterative Proportional Fitting framework predicts unique expert counts with less than 4% error.","excerpt":"We model this effect using a maximum-entropy Iterative Proportional Fitting (IPF) framework that predicts unique expert counts from routing-overlap statistics with less than 4\\% error","support":"source-stated"}],"limitations":["The available evidence is the paper's abstract; it does not provide the experimental setup, detailed results, or the full basis for comparing predicted and observed speedups.","The reported kernel profiling is specifically described for H100; the abstract does not establish whether the findings generalize to other hardware or deployments.","The abstract reports evaluation across sparse MoE models but does not identify their names or give model-by-model results."],"mediaId":"52a229047e6f88ced36d02cd7d7436dcba50185e6acbdbd061c75d6a437aeca8","correctionReason":null,"id":"c0aa900313c4fdaeb65295f95584f5e7cf2553b53fb5ff0a51f6df175d1eecd7","publicationId":"44732b85-407f-426d-8717-832e13ad9bf2","version":1,"sourceUrl":"https://arxiv.org/abs/2610.38981","sourceTitle":"Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-28","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-30T06:45:33.877Z","createdAt":"2026-11-10T19:31:58.625Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"52a229047e6f88ced36d02cd7d7436dcba50185e6acbdbd061c75d6a437aeca8","kind":"article","url":"https://arxiv.org/abs/2610.38981","sourceUrl":"https://arxiv.org/abs/2610.38981","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-30T16:32:28.253Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.38981","checkedAt":"2026-11-10T19:46:10.718Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"52a229047e6f88ced36d02cd7d7436dcba50185e6acbdbd061c75d6a437aeca8","confidence":0.8500000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"d4fdbdd29e3c3d4a9870e10a3147acd3097aeda7ae0855230be3b55d9ee2f1a2","decidedAt":"2026-10-30T16:32:32.108Z"},"provenance":{"textHash":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","observationId":"d5ba7a50062210c739c073a95b52dae42149782d289fe246a2bf093bc7b08ed2","retrievedAt":"2026-10-30T16:32:27.669Z","kind":"publisher-page","preparer":{"model":"server-excerpt-repair/stream-excerpt-repair-v1 (original draft: gpt-6-luna)","runId":"3650bd46-3e38-4be9-8dcd-a8b575f8f13e"},"reviewer":{"model":"gpt-6-luna","runId":"704d3534-e066-45be-8d03-9fe21f57b545"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T19:31:58.625Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Ranjan, Ekagra","url":null,"kind":"unknown"},{"name":"Teru, Komal K.","url":null,"kind":"unknown"},{"name":"Venkitesh, Bharat","url":null,"kind":"unknown"},{"name":"Locatelli, Acyr","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Ranjan, Ekagra","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ranjan, Ekagra","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38981","locator":"head/meta[name=author|citation_author]"},"assertionId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T19:46:10.718Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426","sourceUrl":"https://arxiv.org/abs/2610.38981","contentSha256":"cd3bc13cef0c5b665d1dcf3f7eb779bdc7aa769a775907476a3cecf947a60044","textHash":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","observedAt":"2026-11-10T19:46:10.718Z"}},{"actor":{"name":"Teru, Komal K.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Teru, Komal K.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38981","locator":"head/meta[name=author|citation_author]"},"assertionId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T19:46:10.718Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426","sourceUrl":"https://arxiv.org/abs/2610.38981","contentSha256":"cd3bc13cef0c5b665d1dcf3f7eb779bdc7aa769a775907476a3cecf947a60044","textHash":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","observedAt":"2026-11-10T19:46:10.718Z"}},{"actor":{"name":"Venkitesh, Bharat","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Venkitesh, Bharat","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38981","locator":"head/meta[name=author|citation_author]"},"assertionId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T19:46:10.718Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426","sourceUrl":"https://arxiv.org/abs/2610.38981","contentSha256":"cd3bc13cef0c5b665d1dcf3f7eb779bdc7aa769a775907476a3cecf947a60044","textHash":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","observedAt":"2026-11-10T19:46:10.718Z"}},{"actor":{"name":"Locatelli, Acyr","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Locatelli, Acyr","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38981","locator":"head/meta[name=author|citation_author]"},"assertionId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T19:46:10.718Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426","sourceUrl":"https://arxiv.org/abs/2610.38981","contentSha256":"cd3bc13cef0c5b665d1dcf3f7eb779bdc7aa769a775907476a3cecf947a60044","textHash":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","observedAt":"2026-11-10T19:46:10.718Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.38981","snapshotId":"f0cf34a1c2a075c4e7b70dfcc8a2a8b0aa0184944160d1075e2a590e78cc7426","contentSha256":"cd3bc13cef0c5b665d1dcf3f7eb779bdc7aa769a775907476a3cecf947a60044","textHash":"0dd29b6e186d9a7a578fc17ec18b618cdd42a2026b1bf9cb035f3fcb21cdac9d","observedAt":"2026-11-10T19:46:10.718Z","basis":"publisher-page-metadata","sourceLinks":44,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/c0aa900313c4fdaeb65295f95584f5e7cf2553b53fb5ff0a51f6df175d1eecd7/source"}}}],"audits":[{"publicationId":"44732b85-407f-426d-8717-832e13ad9bf2","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading correctly names the four listed authors, Ekagra Ranjan, Komal K. Teru, Bharat Venkitesh and Acyr Locatelli. It keeps the findings attributed to them as reported results rather than established fact.","excerpt":"Authors:Ekagra Ranjan, Komal K. Teru, Bharat Venkitesh, Acyr Locatelli"},{"dimension":"summary","status":"uncertain","explanation":"The opening sentence says sparse MoE models in general speed up more at low batch sizes. The source limits that finding to recent fine-grained models; smallest repair: say 'recent fine-grained sparse MoE models'. Batch size, the central variable, is never explained (roughly, how many requests are processed together). Fine-grained, L2 and H100 also lack plain-language context. Needed term-library tooltips: batch size, fine-grained MoE, L2 cache, H100. Without these, a non-specialist cannot follow why low versus intermediate batch size matters.","excerpt":"find recent fine-grained models break this trend, showing higher speedup at low batch size"},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-10-28, which matches the v1 submission date on arXiv. The later feed date is correctly not used.","excerpt":"[Submitted on 28 Oct 2026]"},{"dimension":"support","status":"uncertain","explanation":"Claim 3 says the authors attribute the smaller MoE runtime share 'partly' to bandwidth use and L2 behavior; the source says 'mainly'. This weakens the authors' stated attribution, so the smallest repair is to replace 'partly' with 'mainly'. The summary also drops 'substantially' from 'substantially narrows the gap', mildly understating the claim. The closing inference about better estimating speedups is hedged and acceptable. The question about H100 and other models follows from the evidence.","excerpt":"on H100, kernel profiling attributes this mainly to higher bandwidth utilization of MoE kernels relative to attention"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper. It is the most direct access to the source.","excerpt":null}],"createdAt":"2026-11-10T19:31:58.625Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}