{"id":"7bc08abacd862107632d97431bef01f1ccd1e5b656972ccf86c23961e6386e32","title":"Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation","argument":"Authors Hao Cheng, Zhiyu Yang, Bo Ma and Hanzhang Jia compare ways to deactivate experts—components selected by a model to contribute to a task—during inference. They argue that performance depends on whether the weight rule used after deactivation matches the rule used during training. Their experiments compare four implementations on five instruction-tuned checkpoints from four model families, with a base OLMoE checkpoint as an additional comparison. When half the experts are deactivated using activation-based rankings, the implementation with the highest perplexity differs according to the training rule: remaining-expert renormalization for models trained with raw router probabilities, and unnormalized re-routing for models trained with normalized weights. The authors report that retaining the training rule was associated with perplexity from 1.4 to 73 times the intact baseline, while a rule mismatch was associated with 14 to 28,740 times the baseline. Results also vary by task: on OLMoE, GSM8K strict-match accuracy fell to zero under all four implementations. Their weight-sum model further predicts that the perplexity penalty from additional renormalization increases as fewer experts remain active; the abstract says OLMoE experiments support this prediction.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evidence is the arXiv abstract, not the full paper; it does not identify the individual instruction-tuned checkpoints or provide detailed experimental procedures. The abstract reports a prediction supported by OLMoE experiments but does not give the experiment-level results needed to assess that support in detail.","speaker":{"name":"Cheng, Hao, Yang, Zhiyu, Ma, Bo, Jia, Hanzhang","roleAtSource":"Article author"},"source":{"title":"Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.05804","publishedAt":"2026-11-04","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors compare four implementations of inference-time expert deactivation and develop a weight-sum model linking expert contributions and branch scale to the training-time weight rule.","excerpt":"We develop a weight-sum model that relates the loss of expert contributions and changes in branch scale to the weight rule used during training.","url":"https://arxiv.org/abs/2611.05804","relationship":"Source-stated"},{"text":"With half the experts deactivated according to activation-based rankings, the implementation with the highest perplexity differs according to whether training used raw router probabilities or normalized weights.","excerpt":"remaining-expert renormalization yields the highest perplexity in models trained with raw router probabilities, whereas unnormalized re-routing yields the highest perplexity in models trained with normalized weights.","url":"https://arxiv.org/abs/2611.05804","relationship":"Source-stated"},{"text":"The authors report larger perplexity ranges under a rule mismatch than when the training rule is retained.","excerpt":"Perplexity ranges from 1.4 to 73 times the intact baseline when the training rule is retained, and from 14 to 28,740 times the baseline under a rule mismatch.","url":"https://arxiv.org/abs/2611.05804","relationship":"Source-stated"},{"text":"Task outcomes differ by evaluation: on OLMoE, GSM8K strict-match accuracy falls to zero under each of the four implementations.","excerpt":"on OLMoE, GSM8K strict-match accuracy falls to zero under all four implementations.","url":"https://arxiv.org/abs/2611.05804","relationship":"Source-stated"},{"text":"The authors say their model predicts a rising perplexity penalty from additional renormalization as fewer experts are active, and that OLMoE experiments support this prediction.","excerpt":"the perplexity penalty from additional renormalization increases as fewer experts are activated.","url":"https://arxiv.org/abs/2611.05804","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05804","checkedAt":"2026-11-10T17:48:09.939Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well does the weight-sum model predict the effects of expert deactivation on checkpoints beyond those tested?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation","summary":"Authors Hao Cheng, Zhiyu Yang, Bo Ma and Hanzhang Jia compare ways to deactivate experts—components selected by a model to contribute to a task—during inference. They argue that performance depends on whether the weight rule used after deactivation matches the rule used during training. Their experiments compare four implementations on five instruction-tuned checkpoints from four model families, with a base OLMoE checkpoint as an additional comparison. When half the experts are deactivated using activation-based rankings, the implementation with the highest perplexity differs according to the training rule: remaining-expert renormalization for models trained with raw router probabilities, and unnormalized re-routing for models trained with normalized weights. The authors report that retaining the training rule was associated with perplexity from 1.4 to 73 times the intact baseline, while a rule mismatch was associated with 14 to 28,740 times the baseline. Results also vary by task: on OLMoE, GSM8K strict-match accuracy fell to zero under all four implementations. Their weight-sum model further predicts that the perplexity penalty from additional renormalization increases as fewer experts remain active; the abstract says OLMoE experiments support this prediction.","question":"How well does the weight-sum model predict the effects of expert deactivation on checkpoints beyond those tested?","claims":[{"text":"The authors compare four implementations of inference-time expert deactivation and develop a weight-sum model linking expert contributions and branch scale to the training-time weight rule.","excerpt":"We develop a weight-sum model that relates the loss of expert contributions and changes in branch scale to the weight rule used during training.","support":"source-stated"},{"text":"With half the experts deactivated according to activation-based rankings, the implementation with the highest perplexity differs according to whether training used raw router probabilities or normalized weights.","excerpt":"remaining-expert renormalization yields the highest perplexity in models trained with raw router probabilities, whereas unnormalized re-routing yields the highest perplexity in models trained with normalized weights.","support":"source-stated"},{"text":"The authors report larger perplexity ranges under a rule mismatch than when the training rule is retained.","excerpt":"Perplexity ranges from 1.4 to 73 times the intact baseline when the training rule is retained, and from 14 to 28,740 times the baseline under a rule mismatch.","support":"source-stated"},{"text":"Task outcomes differ by evaluation: on OLMoE, GSM8K strict-match accuracy falls to zero under each of the four implementations.","excerpt":"on OLMoE, GSM8K strict-match accuracy falls to zero under all four implementations.","support":"source-stated"},{"text":"The authors say their model predicts a rising perplexity penalty from additional renormalization as fewer experts are active, and that OLMoE experiments support this prediction.","excerpt":"the perplexity penalty from additional renormalization increases as fewer experts are activated.","support":"source-stated"}],"limitations":["The evidence is the arXiv abstract, not the full paper; it does not identify the individual instruction-tuned checkpoints or provide detailed experimental procedures.","The abstract reports a prediction supported by OLMoE experiments but does not give the experiment-level results needed to assess that support in detail."],"mediaId":"15f83de51eec0a07134ab77b53367df735e7de8b24a3c2259be2a9a06f7c62f0","correctionReason":null,"id":"7bc08abacd862107632d97431bef01f1ccd1e5b656972ccf86c23961e6386e32","publicationId":"b1b75592-3142-4db3-8fb4-e9fd6f6f1332","version":1,"sourceUrl":"https://arxiv.org/abs/2611.05804","sourceTitle":"Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-04","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:06:27.674Z","createdAt":"2026-11-10T17:32:39.035Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"15f83de51eec0a07134ab77b53367df735e7de8b24a3c2259be2a9a06f7c62f0","kind":"article","url":"https://arxiv.org/abs/2611.05804","sourceUrl":"https://arxiv.org/abs/2611.05804","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-10T01:54:40.514Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05804","checkedAt":"2026-11-10T17:48:09.939Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"15f83de51eec0a07134ab77b53367df735e7de8b24a3c2259be2a9a06f7c62f0","confidence":0.8500000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"92147710a11fbe5977970008ce429ad5304c6737116245d9b22fcd3900cdcf21","decidedAt":"2026-11-10T01:54:41.782Z"},"provenance":{"textHash":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","observationId":"874ebd82b2b059a16ba606644e1c9b242b885671fa9448f984819a076ece9a96","retrievedAt":"2026-11-10T01:54:40.327Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna; stream-draft-repair-v1","runId":"970a7e09-aaa1-44a2-843d-23e3aa5f631a"},"reviewer":{"model":"gpt-6-luna","runId":"8cfc7a74-e067-4868-894d-e1417cb56502"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T17:32:39.035Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Cheng, Hao","url":null,"kind":"unknown"},{"name":"Yang, Zhiyu","url":null,"kind":"unknown"},{"name":"Ma, Bo","url":null,"kind":"unknown"},{"name":"Jia, Hanzhang","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Cheng, Hao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cheng, Hao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05804","locator":"head/meta[name=author|citation_author]"},"assertionId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T17:48:09.939Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc","sourceUrl":"https://arxiv.org/abs/2611.05804","contentSha256":"97f92f849d117d3d0295f33caadb09a5163a840586dee138d60740e63032ef60","textHash":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","observedAt":"2026-11-10T17:48:09.939Z"}},{"actor":{"name":"Yang, Zhiyu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yang, Zhiyu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05804","locator":"head/meta[name=author|citation_author]"},"assertionId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T17:48:09.939Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc","sourceUrl":"https://arxiv.org/abs/2611.05804","contentSha256":"97f92f849d117d3d0295f33caadb09a5163a840586dee138d60740e63032ef60","textHash":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","observedAt":"2026-11-10T17:48:09.939Z"}},{"actor":{"name":"Ma, Bo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ma, Bo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05804","locator":"head/meta[name=author|citation_author]"},"assertionId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T17:48:09.939Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc","sourceUrl":"https://arxiv.org/abs/2611.05804","contentSha256":"97f92f849d117d3d0295f33caadb09a5163a840586dee138d60740e63032ef60","textHash":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","observedAt":"2026-11-10T17:48:09.939Z"}},{"actor":{"name":"Jia, Hanzhang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jia, Hanzhang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05804","locator":"head/meta[name=author|citation_author]"},"assertionId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T17:48:09.939Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc","sourceUrl":"https://arxiv.org/abs/2611.05804","contentSha256":"97f92f849d117d3d0295f33caadb09a5163a840586dee138d60740e63032ef60","textHash":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","observedAt":"2026-11-10T17:48:09.939Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.05804","snapshotId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc","contentSha256":"97f92f849d117d3d0295f33caadb09a5163a840586dee138d60740e63032ef60","textHash":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","observedAt":"2026-11-10T17:48:09.939Z","basis":"publisher-page-metadata","sourceLinks":43,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/7bc08abacd862107632d97431bef01f1ccd1e5b656972ccf86c23961e6386e32/source"}},"canonicalUrl":"https://escapemeridian.com/readings/matching-the-training-time-weight-rule-in-inference-time-expert-7bc08aba","formats":{"html":"/readings/matching-the-training-time-weight-rule-in-inference-time-expert-7bc08aba","markdown":"/readings/matching-the-training-time-weight-rule-in-inference-time-expert-7bc08aba.md","json":"/readings/matching-the-training-time-weight-rule-in-inference-time-expert-7bc08aba.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[],"entities":[{"id":"time","name":"TIME","kind":"organization","context":"TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.","referenceUrls":["https://time.com/"],"path":"/entities/time","json":"/entities/time.json","markdown":"/entities/time.md"},{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"7bc08abacd862107632d97431bef01f1ccd1e5b656972ccf86c23961e6386e32","revision":"b1b75592-3142-4db3-8fb4-e9fd6f6f1332","revisionKind":"publication-id","title":"Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation","url":"https://escapemeridian.com/readings/matching-the-training-time-weight-rule-in-inference-time-expert-7bc08aba"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.05804","revision":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","revisionKind":"sha256-source-text","title":"Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation","url":"https://arxiv.org/abs/2611.05804"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"7bc08abacd862107632d97431bef01f1ccd1e5b656972ccf86c23961e6386e32","title":"Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation","argument":"Authors Hao Cheng, Zhiyu Yang, Bo Ma and Hanzhang Jia compare ways to deactivate experts—components selected by a model to contribute to a task—during inference. They argue that performance depends on whether the weight rule used after deactivation matches the rule used during training. Their experiments compare four implementations on five instruction-tuned checkpoints from four model families, with a base OLMoE checkpoint as an additional comparison. When half the experts are deactivated using activation-based rankings, the implementation with the highest perplexity differs according to the training rule: remaining-expert renormalization for models trained with raw router probabilities, and unnormalized re-routing for models trained with normalized weights. The authors report that retaining the training rule was associated with perplexity from 1.4 to 73 times the intact baseline, while a rule mismatch was associated with 14 to 28,740 times the baseline. Results also vary by task: on OLMoE, GSM8K strict-match accuracy fell to zero under all four implementations. Their weight-sum model further predicts that the perplexity penalty from additional renormalization increases as fewer experts remain active; the abstract says OLMoE experiments support this prediction.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evidence is the arXiv abstract, not the full paper; it does not identify the individual instruction-tuned checkpoints or provide detailed experimental procedures. The abstract reports a prediction supported by OLMoE experiments but does not give the experiment-level results needed to assess that support in detail.","speaker":{"name":"Cheng, Hao, Yang, Zhiyu, Ma, Bo, Jia, Hanzhang","roleAtSource":"Article author"},"source":{"title":"Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.05804","publishedAt":"2026-11-04","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors compare four implementations of inference-time expert deactivation and develop a weight-sum model linking expert contributions and branch scale to the training-time weight rule.","excerpt":"We develop a weight-sum model that relates the loss of expert contributions and changes in branch scale to the weight rule used during training.","url":"https://arxiv.org/abs/2611.05804","relationship":"Source-stated"},{"text":"With half the experts deactivated according to activation-based rankings, the implementation with the highest perplexity differs according to whether training used raw router probabilities or normalized weights.","excerpt":"remaining-expert renormalization yields the highest perplexity in models trained with raw router probabilities, whereas unnormalized re-routing yields the highest perplexity in models trained with normalized weights.","url":"https://arxiv.org/abs/2611.05804","relationship":"Source-stated"},{"text":"The authors report larger perplexity ranges under a rule mismatch than when the training rule is retained.","excerpt":"Perplexity ranges from 1.4 to 73 times the intact baseline when the training rule is retained, and from 14 to 28,740 times the baseline under a rule mismatch.","url":"https://arxiv.org/abs/2611.05804","relationship":"Source-stated"},{"text":"Task outcomes differ by evaluation: on OLMoE, GSM8K strict-match accuracy falls to zero under each of the four implementations.","excerpt":"on OLMoE, GSM8K strict-match accuracy falls to zero under all four implementations.","url":"https://arxiv.org/abs/2611.05804","relationship":"Source-stated"},{"text":"The authors say their model predicts a rising perplexity penalty from additional renormalization as fewer experts are active, and that OLMoE experiments support this prediction.","excerpt":"the perplexity penalty from additional renormalization increases as fewer experts are activated.","url":"https://arxiv.org/abs/2611.05804","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05804","checkedAt":"2026-11-10T17:48:09.939Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well does the weight-sum model predict the effects of expert deactivation on checkpoints beyond those tested?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation","summary":"Authors Hao Cheng, Zhiyu Yang, Bo Ma and Hanzhang Jia compare ways to deactivate experts—components selected by a model to contribute to a task—during inference. They argue that performance depends on whether the weight rule used after deactivation matches the rule used during training. Their experiments compare four implementations on five instruction-tuned checkpoints from four model families, with a base OLMoE checkpoint as an additional comparison. When half the experts are deactivated using activation-based rankings, the implementation with the highest perplexity differs according to the training rule: remaining-expert renormalization for models trained with raw router probabilities, and unnormalized re-routing for models trained with normalized weights. The authors report that retaining the training rule was associated with perplexity from 1.4 to 73 times the intact baseline, while a rule mismatch was associated with 14 to 28,740 times the baseline. Results also vary by task: on OLMoE, GSM8K strict-match accuracy fell to zero under all four implementations. Their weight-sum model further predicts that the perplexity penalty from additional renormalization increases as fewer experts remain active; the abstract says OLMoE experiments support this prediction.","question":"How well does the weight-sum model predict the effects of expert deactivation on checkpoints beyond those tested?","claims":[{"text":"The authors compare four implementations of inference-time expert deactivation and develop a weight-sum model linking expert contributions and branch scale to the training-time weight rule.","excerpt":"We develop a weight-sum model that relates the loss of expert contributions and changes in branch scale to the weight rule used during training.","support":"source-stated"},{"text":"With half the experts deactivated according to activation-based rankings, the implementation with the highest perplexity differs according to whether training used raw router probabilities or normalized weights.","excerpt":"remaining-expert renormalization yields the highest perplexity in models trained with raw router probabilities, whereas unnormalized re-routing yields the highest perplexity in models trained with normalized weights.","support":"source-stated"},{"text":"The authors report larger perplexity ranges under a rule mismatch than when the training rule is retained.","excerpt":"Perplexity ranges from 1.4 to 73 times the intact baseline when the training rule is retained, and from 14 to 28,740 times the baseline under a rule mismatch.","support":"source-stated"},{"text":"Task outcomes differ by evaluation: on OLMoE, GSM8K strict-match accuracy falls to zero under each of the four implementations.","excerpt":"on OLMoE, GSM8K strict-match accuracy falls to zero under all four implementations.","support":"source-stated"},{"text":"The authors say their model predicts a rising perplexity penalty from additional renormalization as fewer experts are active, and that OLMoE experiments support this prediction.","excerpt":"the perplexity penalty from additional renormalization increases as fewer experts are activated.","support":"source-stated"}],"limitations":["The evidence is the arXiv abstract, not the full paper; it does not identify the individual instruction-tuned checkpoints or provide detailed experimental procedures.","The abstract reports a prediction supported by OLMoE experiments but does not give the experiment-level results needed to assess that support in detail."],"mediaId":"15f83de51eec0a07134ab77b53367df735e7de8b24a3c2259be2a9a06f7c62f0","correctionReason":null,"id":"7bc08abacd862107632d97431bef01f1ccd1e5b656972ccf86c23961e6386e32","publicationId":"b1b75592-3142-4db3-8fb4-e9fd6f6f1332","version":1,"sourceUrl":"https://arxiv.org/abs/2611.05804","sourceTitle":"Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-04","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:06:27.674Z","createdAt":"2026-11-10T17:32:39.035Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"15f83de51eec0a07134ab77b53367df735e7de8b24a3c2259be2a9a06f7c62f0","kind":"article","url":"https://arxiv.org/abs/2611.05804","sourceUrl":"https://arxiv.org/abs/2611.05804","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-10T01:54:40.514Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05804","checkedAt":"2026-11-10T17:48:09.939Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"15f83de51eec0a07134ab77b53367df735e7de8b24a3c2259be2a9a06f7c62f0","confidence":0.8500000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"92147710a11fbe5977970008ce429ad5304c6737116245d9b22fcd3900cdcf21","decidedAt":"2026-11-10T01:54:41.782Z"},"provenance":{"textHash":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","observationId":"874ebd82b2b059a16ba606644e1c9b242b885671fa9448f984819a076ece9a96","retrievedAt":"2026-11-10T01:54:40.327Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna; stream-draft-repair-v1","runId":"970a7e09-aaa1-44a2-843d-23e3aa5f631a"},"reviewer":{"model":"gpt-6-luna","runId":"8cfc7a74-e067-4868-894d-e1417cb56502"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T17:32:39.035Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Cheng, Hao","url":null,"kind":"unknown"},{"name":"Yang, Zhiyu","url":null,"kind":"unknown"},{"name":"Ma, Bo","url":null,"kind":"unknown"},{"name":"Jia, Hanzhang","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Cheng, Hao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cheng, Hao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05804","locator":"head/meta[name=author|citation_author]"},"assertionId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T17:48:09.939Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc","sourceUrl":"https://arxiv.org/abs/2611.05804","contentSha256":"97f92f849d117d3d0295f33caadb09a5163a840586dee138d60740e63032ef60","textHash":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","observedAt":"2026-11-10T17:48:09.939Z"}},{"actor":{"name":"Yang, Zhiyu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yang, Zhiyu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05804","locator":"head/meta[name=author|citation_author]"},"assertionId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T17:48:09.939Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc","sourceUrl":"https://arxiv.org/abs/2611.05804","contentSha256":"97f92f849d117d3d0295f33caadb09a5163a840586dee138d60740e63032ef60","textHash":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","observedAt":"2026-11-10T17:48:09.939Z"}},{"actor":{"name":"Ma, Bo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ma, Bo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05804","locator":"head/meta[name=author|citation_author]"},"assertionId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T17:48:09.939Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc","sourceUrl":"https://arxiv.org/abs/2611.05804","contentSha256":"97f92f849d117d3d0295f33caadb09a5163a840586dee138d60740e63032ef60","textHash":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","observedAt":"2026-11-10T17:48:09.939Z"}},{"actor":{"name":"Jia, Hanzhang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jia, Hanzhang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05804","locator":"head/meta[name=author|citation_author]"},"assertionId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T17:48:09.939Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc","sourceUrl":"https://arxiv.org/abs/2611.05804","contentSha256":"97f92f849d117d3d0295f33caadb09a5163a840586dee138d60740e63032ef60","textHash":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","observedAt":"2026-11-10T17:48:09.939Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.05804","snapshotId":"f355e35520f4a71915868fb0beebcdfd0c04ad2ab9196765d8edb52eaed96fbc","contentSha256":"97f92f849d117d3d0295f33caadb09a5163a840586dee138d60740e63032ef60","textHash":"17812c8f11bd61ca152bd99094b8a288fe5240a26f4ebc4a9e3c023ca4443431","observedAt":"2026-11-10T17:48:09.939Z","basis":"publisher-page-metadata","sourceLinks":43,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/7bc08abacd862107632d97431bef01f1ccd1e5b656972ccf86c23961e6386e32/source"}}}],"audits":[{"publicationId":"b1b75592-3142-4db3-8fb4-e9fd6f6f1332","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading names the four listed authors and frames the findings as their comparisons, model and reported results. This matches the arXiv abstract page.","excerpt":"Authors:Hao Cheng (Zhejiang Gongshang University), Zhiyu Yang (Chang'an University), Bo Ma (Zhejiang Gongshang University), Hanzhang Jia (University of Sussex)"},{"dimension":"summary","status":"uncertain","explanation":"The summary follows the abstract faithfully, but a non-specialist would struggle with it. Central terms are unexplained: perplexity, renormalization, raw router probabilities versus normalized weights, checkpoints, OLMoE and GSM8K. Readers cannot tell whether higher perplexity means worse performance or why the results matter. The added gloss of experts as components selected 'to contribute to a task' is also loose; the source describes routing per token. Smallest repair: add term-library tooltips for these terms, state that higher perplexity indicates worse prediction, and revise the expert gloss to refer to token routing in a mixture-of-experts model. The question is fair.","excerpt":"Expert deactivation at inference time changes both token routing and expert weights, and its effect on performance depends on how these changes are implemented."},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-11-04, which matches the arXiv v1 submission date. It is not the later feed date of 2026-11-05.","excerpt":"[Submitted on 4 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim is directly supported by the abstract, including the four implementations, the perplexity ranges, GSM8K going to zero on OLMoE, and the prediction supported by OLMoE. The draft makes no unsupported benefits or forecasts. The limitations correctly note that the evidence is an abstract only.","excerpt":"Perplexity ranges from 1.4 to 73 times the intact baseline when the training rule is retained, and from 14 to 28,740 times the baseline under a rule mismatch."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper, which is the source itself.","excerpt":null}],"createdAt":"2026-11-10T17:32:39.035Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}