Cheng, Hao, Yang, Zhiyu, Ma, Bo, Jia, Hanzhang · Nov 4, 2026
Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation
Markdown · JSON · Original source
Authors Hao Cheng, Zhiyu Yang, Bo Ma and Hanzhang Jia compare ways to deactivate experts—components selected by a model to contribute to a task—during inference. They argue that performance depends on whether the weight rule used after deactivation matches the rule used during training. Their experiments compare four implementations on five instruction-tuned checkpoints from four model families, with a base OLMoE checkpoint as an additional comparison. When half the experts are deactivated using activation-based rankings, the implementation with the highest perplexity differs according to the training rule: remaining-expert renormalization for models trained with raw router probabilities, and unnormalized re-routing for models trained with normalized weights. The authors report that retaining the training rule was associated with perplexity from 1.4 to 73 times the intact baseline, while a rule mismatch was associated with 14 to 28,740 times the baseline. Results also vary by task: on OLMoE, GSM8K strict-match accuracy fell to zero under all four implementations. Their weight-sum model further predicts that the perplexity penalty from additional renormalization increases as fewer experts remain active; the abstract says OLMoE experiments support this prediction.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 4, 2026 (2026-11-04).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T17:32:39.035Z. First observed: 2026-11-05T05:06:27.674Z. Retrieved: 2026-11-10T01:54:40.327Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The evidence is the abstract, not the full paper; it does not identify the individual instruction-tuned checkpoints or provide detailed experimental procedures. The abstract reports a prediction supported by OLMoE experiments but does not give the experiment-level results needed to assess that support in detail.
Open question
How well does the weight-sum model predict the effects of expert deactivation on checkpoints beyond those tested?
Source-stated
The authors compare four implementations of inference-time expert deactivation and develop a weight-sum model linking expert contributions and branch scale to the training-time weight rule.
We develop a weight-sum model that relates the loss of expert contributions and changes in branch scale to the weight rule used during training.
Source-stated
With half the experts deactivated according to activation-based rankings, the implementation with the highest perplexity differs according to whether training used raw router probabilities or normalized weights.
remaining-expert renormalization yields the highest perplexity in models trained with raw router probabilities, whereas unnormalized re-routing yields the highest perplexity in models trained with normalized weights.
Source-stated
The authors report larger perplexity ranges under a rule mismatch than when the training rule is retained.
Perplexity ranges from 1.4 to 73 times the intact baseline when the training rule is retained, and from 14 to 28,740 times the baseline under a rule mismatch.
Source-stated
Task outcomes differ by evaluation: on OLMoE, GSM8K strict-match accuracy falls to zero under each of the four implementations.
on OLMoE, GSM8K strict-match accuracy falls to zero under all four implementations.
Source-stated
The authors say their model predicts a rising perplexity penalty from additional renormalization as fewer experts are active, and that OLMoE experiments support this prediction.
the perplexity penalty from additional renormalization increases as fewer experts are activated.
Source and contributor context
Cheng, Hao, Yang, Zhiyu, Ma, Bo, Jia, Hanzhang · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: b1b75592-3142-4db3-8fb4-e9fd6f6f1332.
Prepared by gpt-6-luna; stream-draft-repair-v1; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T17:32:39.035Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 874ebd82b2b059a16ba606644e1c9b242b885671fa9448f984819a076ece9a96. Source text hash: 17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431.
Revision and source-check history
Version 1 · 2026-11-10T17:32:39.035Z
Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation
Authors Hao Cheng, Zhiyu Yang, Bo Ma and Hanzhang Jia compare ways to deactivate experts—components selected by a model to contribute to a task—during inference. They argue that performance depends on whether the weight rule used after deactivation matches the rule used during training. Their experiments compare four implementations on five instruction-tuned checkpoints from four model families, with a base OLMoE checkpoint as an additional comparison. When half the experts are deactivated using activation-based rankings, the implementation with the highest perplexity differs according to the training rule: remaining-expert renormalization for models trained with raw router probabilities, and unnormalized re-routing for models trained with normalized weights. The authors report that retaining the training rule was associated with perplexity from 1.4 to 73 times the intact baseline, while a rule mismatch was associated with 14 to 28,740 times the baseline. Results also vary by task: on OLMoE, GSM8K strict-match accuracy fell to zero under all four implementations. Their weight-sum model further predicts that the perplexity penalty from additional renormalization increases as fewer experts remain active; the abstract says OLMoE experiments support this prediction.
Original source
cs.AI · Article
Source details
Original source date: Nov 4, 2026 (2026-11-04).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T17:32:39.035Z. First observed: 2026-11-05T05:06:27.674Z. Retrieved: 2026-11-10T01:54:40.327Z.
Limits
The evidence is the abstract, not the full paper; it does not identify the individual instruction-tuned checkpoints or provide detailed experimental procedures. The abstract reports a prediction supported by OLMoE experiments but does not give the experiment-level results needed to assess that support in detail.
Open question
How well does the weight-sum model predict the effects of expert deactivation on checkpoints beyond those tested?
Source-stated
The authors compare four implementations of inference-time expert deactivation and develop a weight-sum model linking expert contributions and branch scale to the training-time weight rule.
We develop a weight-sum model that relates the loss of expert contributions and changes in branch scale to the weight rule used during training.
Source-stated
With half the experts deactivated according to activation-based rankings, the implementation with the highest perplexity differs according to whether training used raw router probabilities or normalized weights.
remaining-expert renormalization yields the highest perplexity in models trained with raw router probabilities, whereas unnormalized re-routing yields the highest perplexity in models trained with normalized weights.
Source-stated
The authors report larger perplexity ranges under a rule mismatch than when the training rule is retained.
Perplexity ranges from 1.4 to 73 times the intact baseline when the training rule is retained, and from 14 to 28,740 times the baseline under a rule mismatch.
Source-stated
Task outcomes differ by evaluation: on OLMoE, GSM8K strict-match accuracy falls to zero under each of the four implementations.
on OLMoE, GSM8K strict-match accuracy falls to zero under all four implementations.
Source-stated
The authors say their model predicts a rising perplexity penalty from additional renormalization as fewer experts are active, and that OLMoE experiments support this prediction.
the perplexity penalty from additional renormalization increases as fewer experts are activated.
Source and contributor context
Cheng, Hao, Yang, Zhiyu, Ma, Bo, Jia, Hanzhang · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: b1b75592-3142-4db3-8fb4-e9fd6f6f1332.
Prepared by gpt-6-luna; stream-draft-repair-v1; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T17:32:39.035Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 874ebd82b2b059a16ba606644e1c9b242b885671fa9448f984819a076ece9a96. Source text hash: 17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431.
uncertain · 2026-11-10T17:32:39.035Z · anthropic/claude-opus-5.5
Publication: b1b75592-3142-4db3-8fb4-e9fd6f6f1332
attribution — pass: The reading names the four listed authors and frames the findings as their comparisons, model and reported results. This matches the arXiv abstract page.
Authors:Hao Cheng (Zhejiang Gongshang University), Zhiyu Yang (Chang'an University), Bo Ma (Zhejiang Gongshang University), Hanzhang Jia (University of Sussex)
summary — uncertain: The summary follows the abstract faithfully, but a non-specialist would struggle with it. Central terms are unexplained: perplexity, renormalization, raw router probabilities versus normalized weights, checkpoints, OLMoE and GSM8K. Readers cannot tell whether higher perplexity means worse performance or why the results matter. The added gloss of experts as components selected 'to contribute to a task' is also loose; the source describes routing per token. Smallest repair: add term-library tooltips for these terms, state that higher perplexity indicates worse prediction, and revise the expert gloss to refer to token routing in a mixture-of-experts model. The question is fair.
Expert deactivation at inference time changes both token routing and expert weights, and its effect on performance depends on how these changes are implemented.
date — pass: originalPublished is 2026-11-04, which matches the arXiv v1 submission date. It is not the later feed date of 2026-11-05.
[Submitted on 4 Nov 2026]
support — pass: Each claim is directly supported by the abstract, including the four implementations, the perplexity ranges, GSM8K going to zero on OLMoE, and the prediction supported by OLMoE. The draft makes no unsupported benefits or forecasts. The limitations correctly note that the evidence is an abstract only.
Perplexity ranges from 1.4 to 73 times the intact baseline when the training rule is retained, and from 14 to 28,740 times the baseline under a rule mismatch.
media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the source itself.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv