{"id":"ab90bf4c83956c6de48ff82f4b5aecde8912e50425298db52fe1bf38be68c4cd","title":"Your Experts Have Partners. There's HOPE for Keeping Them Together","argument":"Amazon author Alex M. Tseng presents HOPE, a method for pruning experts from Mixture-of-Experts (MoE) language models. Unlike first-order methods, which score experts independently, HOPE accounts for pairwise interactions: experts that often activate together may form cooperative groups, so removing one can impair the contribution of another. The authors say this matters most when pruning aggressively, because more cooperative links may be broken.\n\nThe post reports evaluations on three models, two calibration datasets and several benchmarks. Across the tested conditions, HOPE achieved the best average rank among five methods; at 50% pruning, its average rank was 1.58 versus 2.42 for the next-best method. On Qwen3.5-35B-A3B at that pruning rate, the authors report a score of 42.3 on SWE-Bench Pro, compared with 41.8 for the unpruned model and 36.2 for REAP.\n\nThe authors describe the gains as most pronounced at 40–50% pruning, while first-order methods are sometimes competitive at 10–20%. HOPE accounts for pairwise interactions within each layer, not cross-layer interactions; the authors say their cross-layer results remain preliminary. How would HOPE’s results change if it accounted for cross-layer interactions or interactions among three or more experts?","topicIds":["intelligence-and-discovery","work-and-agency"],"domain":"work","claimType":"attributed reading","limits":"The reported results cover three models, two calibration datasets and the benchmarks described in the post; they do not establish performance beyond those evaluations. The authors say HOPE's advantage shrinks at lower pruning rates, where first-order methods are sometimes competitive. HOPE does not account for cross-layer interactions, and the authors describe their cross-layer results as preliminary. HOPE models pairwise interactions; the post says interactions among three or more experts could provide additional signal but are difficult to estimate.","speaker":{"name":"Hugging Face","roleAtSource":"Publisher"},"source":{"title":"Your Experts Have Partners. There's HOPE for Keeping Them Together","publisher":"Hugging Face","format":"Article","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","publishedAt":"2026-11-09T12:34:29.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Alex M. Tseng presents HOPE, a pruning method that accounts for pairwise expert interactions in Mixture-of-Experts models.","excerpt":"Our newest paper derives HOPE (Higher-Order Pruning of Experts), a second-order method that accounts for pairwise expert interactions.","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"},{"text":"The authors report HOPE achieved the best average rank across the tested models, calibration sets and benchmarks.","excerpt":"Across all conditions, HOPE achieved the best average rank (2.07 out of 5 methods)","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"},{"text":"At 50% pruning, the authors report HOPE's average rank was 1.58, compared with 2.42 for REAP.","excerpt":"At 50% pruning, HOPE's average rank improves to 1.58 (vs. 2.42 for the next-best method, REAP).","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"},{"text":"On Qwen3.5-35B-A3B at 50% pruning, the authors report HOPE scored 42.3 on SWE-Bench Pro, versus 41.8 for the unpruned model and 36.2 for REAP.","excerpt":"On Qwen3.5-35B-A3B at 50% pruning, HOPE achieved 42.3 on SWE-Bench Pro, matching the unpruned model's 41.8 (while REAP dropped to 36.2, a 13% regression).","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"},{"text":"The authors say HOPE's advantage is most pronounced at 40–50% pruning; at 10–20%, first-order methods are sometimes competitive.","excerpt":"HOPE's advantage is most pronounced at higher pruning rates (40–50%). At lower pruning rates (10–20%), first-order methods like REAP and MAN are sometimes competitive","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"},{"text":"HOPE prunes layers independently and does not account for cross-layer interactions; the authors describe their cross-layer results as preliminary.","excerpt":"HOPE prunes each layer independently and does not account for cross-layer interactions. In our paper, we presented a theoretical framework for cross-layer HOPE, but the results remain preliminary.","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"},{"text":"HOPE captures pairwise interactions, while interactions among three or more experts could provide additional signal, according to the authors.","excerpt":"Finally, HOPE captures pairwise interactions, but even higher-order interactions (triplets and beyond) could provide additional signal","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"}],"sourceMedia":{"status":"found","sourceUrl":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","checkedAt":"2026-11-10T17:34:11.199Z","items":[],"note":"Media was declared by the source, but no asset passed the supported availability checks. Open the original source to view it."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How would HOPE’s results change if it accounted for cross-layer interactions or interactions among three or more experts?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Your Experts Have Partners. There's HOPE for Keeping Them Together","summary":"Amazon author Alex M. Tseng presents HOPE, a method for pruning experts from Mixture-of-Experts (MoE) language models. Unlike first-order methods, which score experts independently, HOPE accounts for pairwise interactions: experts that often activate together may form cooperative groups, so removing one can impair the contribution of another. The authors say this matters most when pruning aggressively, because more cooperative links may be broken.\n\nThe post reports evaluations on three models, two calibration datasets and several benchmarks. Across the tested conditions, HOPE achieved the best average rank among five methods; at 50% pruning, its average rank was 1.58 versus 2.42 for the next-best method. On Qwen3.5-35B-A3B at that pruning rate, the authors report a score of 42.3 on SWE-Bench Pro, compared with 41.8 for the unpruned model and 36.2 for REAP.\n\nThe authors describe the gains as most pronounced at 40–50% pruning, while first-order methods are sometimes competitive at 10–20%. HOPE accounts for pairwise interactions within each layer, not cross-layer interactions; the authors say their cross-layer results remain preliminary. How would HOPE’s results change if it accounted for cross-layer interactions or interactions among three or more experts?","question":"How would HOPE’s results change if it accounted for cross-layer interactions or interactions among three or more experts?","claims":[{"text":"Alex M. Tseng presents HOPE, a pruning method that accounts for pairwise expert interactions in Mixture-of-Experts models.","excerpt":"Our newest paper derives HOPE (Higher-Order Pruning of Experts), a second-order method that accounts for pairwise expert interactions.","support":"source-stated"},{"text":"The authors report HOPE achieved the best average rank across the tested models, calibration sets and benchmarks.","excerpt":"Across all conditions, HOPE achieved the best average rank (2.07 out of 5 methods)","support":"source-stated"},{"text":"At 50% pruning, the authors report HOPE's average rank was 1.58, compared with 2.42 for REAP.","excerpt":"At 50% pruning, HOPE's average rank improves to 1.58 (vs. 2.42 for the next-best method, REAP).","support":"source-stated"},{"text":"On Qwen3.5-35B-A3B at 50% pruning, the authors report HOPE scored 42.3 on SWE-Bench Pro, versus 41.8 for the unpruned model and 36.2 for REAP.","excerpt":"On Qwen3.5-35B-A3B at 50% pruning, HOPE achieved 42.3 on SWE-Bench Pro, matching the unpruned model's 41.8 (while REAP dropped to 36.2, a 13% regression).","support":"source-stated"},{"text":"The authors say HOPE's advantage is most pronounced at 40–50% pruning; at 10–20%, first-order methods are sometimes competitive.","excerpt":"HOPE's advantage is most pronounced at higher pruning rates (40–50%). At lower pruning rates (10–20%), first-order methods like REAP and MAN are sometimes competitive","support":"source-stated"},{"text":"HOPE prunes layers independently and does not account for cross-layer interactions; the authors describe their cross-layer results as preliminary.","excerpt":"HOPE prunes each layer independently and does not account for cross-layer interactions. In our paper, we presented a theoretical framework for cross-layer HOPE, but the results remain preliminary.","support":"source-stated"},{"text":"HOPE captures pairwise interactions, while interactions among three or more experts could provide additional signal, according to the authors.","excerpt":"Finally, HOPE captures pairwise interactions, but even higher-order interactions (triplets and beyond) could provide additional signal","support":"source-stated"}],"limitations":["The reported results cover three models, two calibration datasets and the benchmarks described in the post; they do not establish performance beyond those evaluations.","The authors say HOPE's advantage shrinks at lower pruning rates, where first-order methods are sometimes competitive.","HOPE does not account for cross-layer interactions, and the authors describe their cross-layer results as preliminary.","HOPE models pairwise interactions; the post says interactions among three or more experts could provide additional signal but are difficult to estimate."],"mediaId":"55d059b88c6c576a2f3d9ab40518505078744d93ab6e7c4bc3d7b8ab49056416","correctionReason":null,"id":"ab90bf4c83956c6de48ff82f4b5aecde8912e50425298db52fe1bf38be68c4cd","publicationId":"744e4ab7-f45d-498a-8a6b-9461b7e96541","version":1,"sourceUrl":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","sourceTitle":"Your Experts Have Partners. There's HOPE for Keeping Them Together","publisher":"Hugging Face","publishedAt":"2026-11-09T12:34:29.000Z","originalPublished":{"value":"2026-11-09T12:34:29.000Z","precision":"instant","basis":"feed_entry"},"firstObservedAt":"2026-11-09T12:36:33.153Z","createdAt":"2026-11-10T17:03:59.661Z","historical":false,"entityIds":["hugging-face"],"topicIds":["intelligence-and-discovery","work-and-agency"],"media":[{"id":"55d059b88c6c576a2f3d9ab40518505078744d93ab6e7c4bc3d7b8ab49056416","kind":"article","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","sourceUrl":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T12:45:41.117Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"found","sourceUrl":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","checkedAt":"2026-11-10T17:34:11.199Z","items":[],"note":"Media was declared by the source, but no asset passed the supported availability checks. Open the original source to view it."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"55d059b88c6c576a2f3d9ab40518505078744d93ab6e7c4bc3d7b8ab49056416","confidence":0.99,"provider":"typesafe","model":"jev-1.13.0","inputHash":"72ab8b99ac0c43c82ea48ee74d941f87b706b6864f2b6b2b179fe4604e154f2f","decidedAt":"2026-11-09T12:45:41.996Z"},"provenance":{"textHash":"74d15c233a49ec339d9b1eee5a02633ec8c8406e33d7c881b3532315f4c67f61","observationId":"b65c78caee9e22b49fc9227c418eaf1caca12e369ca29818a159c67d4222911f","retrievedAt":"2026-11-09T12:45:40.946Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna; stream-draft-repair-v1","runId":"6301b998-8bc9-44d2-8092-3f134e1a9113"},"reviewer":{"model":"gpt-6-luna","runId":"c075cccc-9548-43f3-8a3e-250d9d27ad2d"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T17:03:59.661Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[],"authorRole":"article-author","hosts":[],"status":"missing","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[],"participationCoverage":{"total":0,"returned":0,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","snapshotId":"8fdd8a1213d4066302b5e5ceb9b41efdea93b248307559b3d71851afbf386bae","contentSha256":"551848b2c414602d347bf0070e451066c0446e11c7904ec2f93b52d968648b8e","textHash":"74d15c233a49ec339d9b1eee5a02633ec8c8406e33d7c881b3532315f4c67f61","observedAt":"2026-11-09T12:45:40.946Z","basis":"publisher-page-metadata","sourceLinks":42,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/ab90bf4c83956c6de48ff82f4b5aecde8912e50425298db52fe1bf38be68c4cd/source"}},"canonicalUrl":"https://escapemeridian.com/readings/your-experts-have-partners-there-s-hope-for-keeping-them-ab90bf4c","formats":{"html":"/readings/your-experts-have-partners-there-s-hope-for-keeping-them-ab90bf4c","markdown":"/readings/your-experts-have-partners-there-s-hope-for-keeping-them-ab90bf4c.md","json":"/readings/your-experts-have-partners-there-s-hope-for-keeping-them-ab90bf4c.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-mixture-of-experts","name":"Mixture of experts","explanation":"A model design that sends each input to a few specialized sub-networks instead of the whole model, which saves computation.","referenceUrl":"https://developers.google.com/machine-learning/glossary#mixture-of-experts"},{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"hugging-face","name":"Hugging Face","kind":"organization","context":"Hugging Face hosts models, datasets and software that let people share, adapt and inspect AI systems.","referenceUrls":["https://huggingface.co/blog"],"path":"/entities/hugging-face","json":"/entities/hugging-face.json","markdown":"/entities/hugging-face.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"ab90bf4c83956c6de48ff82f4b5aecde8912e50425298db52fe1bf38be68c4cd","revision":"744e4ab7-f45d-498a-8a6b-9461b7e96541","revisionKind":"publication-id","title":"Your Experts Have Partners. There's HOPE for Keeping Them Together","url":"https://escapemeridian.com/readings/your-experts-have-partners-there-s-hope-for-keeping-them-ab90bf4c"},"source":{"type":"source","id":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","revision":"74d15c233a49ec339d9b1eee5a02633ec8c8406e33d7c881b3532315f4c67f61","revisionKind":"sha256-source-text","title":"Your Experts Have Partners. There's HOPE for Keeping Them Together","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"ab90bf4c83956c6de48ff82f4b5aecde8912e50425298db52fe1bf38be68c4cd","title":"Your Experts Have Partners. There's HOPE for Keeping Them Together","argument":"Amazon author Alex M. Tseng presents HOPE, a method for pruning experts from Mixture-of-Experts (MoE) language models. Unlike first-order methods, which score experts independently, HOPE accounts for pairwise interactions: experts that often activate together may form cooperative groups, so removing one can impair the contribution of another. The authors say this matters most when pruning aggressively, because more cooperative links may be broken.\n\nThe post reports evaluations on three models, two calibration datasets and several benchmarks. Across the tested conditions, HOPE achieved the best average rank among five methods; at 50% pruning, its average rank was 1.58 versus 2.42 for the next-best method. On Qwen3.5-35B-A3B at that pruning rate, the authors report a score of 42.3 on SWE-Bench Pro, compared with 41.8 for the unpruned model and 36.2 for REAP.\n\nThe authors describe the gains as most pronounced at 40–50% pruning, while first-order methods are sometimes competitive at 10–20%. HOPE accounts for pairwise interactions within each layer, not cross-layer interactions; the authors say their cross-layer results remain preliminary. How would HOPE’s results change if it accounted for cross-layer interactions or interactions among three or more experts?","topicIds":["intelligence-and-discovery","work-and-agency"],"domain":"work","claimType":"attributed reading","limits":"The reported results cover three models, two calibration datasets and the benchmarks described in the post; they do not establish performance beyond those evaluations. The authors say HOPE's advantage shrinks at lower pruning rates, where first-order methods are sometimes competitive. HOPE does not account for cross-layer interactions, and the authors describe their cross-layer results as preliminary. HOPE models pairwise interactions; the post says interactions among three or more experts could provide additional signal but are difficult to estimate.","speaker":{"name":"Hugging Face","roleAtSource":"Publisher"},"source":{"title":"Your Experts Have Partners. There's HOPE for Keeping Them Together","publisher":"Hugging Face","format":"Article","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","publishedAt":"2026-11-09T12:34:29.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Alex M. Tseng presents HOPE, a pruning method that accounts for pairwise expert interactions in Mixture-of-Experts models.","excerpt":"Our newest paper derives HOPE (Higher-Order Pruning of Experts), a second-order method that accounts for pairwise expert interactions.","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"},{"text":"The authors report HOPE achieved the best average rank across the tested models, calibration sets and benchmarks.","excerpt":"Across all conditions, HOPE achieved the best average rank (2.07 out of 5 methods)","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"},{"text":"At 50% pruning, the authors report HOPE's average rank was 1.58, compared with 2.42 for REAP.","excerpt":"At 50% pruning, HOPE's average rank improves to 1.58 (vs. 2.42 for the next-best method, REAP).","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"},{"text":"On Qwen3.5-35B-A3B at 50% pruning, the authors report HOPE scored 42.3 on SWE-Bench Pro, versus 41.8 for the unpruned model and 36.2 for REAP.","excerpt":"On Qwen3.5-35B-A3B at 50% pruning, HOPE achieved 42.3 on SWE-Bench Pro, matching the unpruned model's 41.8 (while REAP dropped to 36.2, a 13% regression).","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"},{"text":"The authors say HOPE's advantage is most pronounced at 40–50% pruning; at 10–20%, first-order methods are sometimes competitive.","excerpt":"HOPE's advantage is most pronounced at higher pruning rates (40–50%). At lower pruning rates (10–20%), first-order methods like REAP and MAN are sometimes competitive","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"},{"text":"HOPE prunes layers independently and does not account for cross-layer interactions; the authors describe their cross-layer results as preliminary.","excerpt":"HOPE prunes each layer independently and does not account for cross-layer interactions. In our paper, we presented a theoretical framework for cross-layer HOPE, but the results remain preliminary.","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"},{"text":"HOPE captures pairwise interactions, while interactions among three or more experts could provide additional signal, according to the authors.","excerpt":"Finally, HOPE captures pairwise interactions, but even higher-order interactions (triplets and beyond) could provide additional signal","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","relationship":"Source-stated"}],"sourceMedia":{"status":"found","sourceUrl":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","checkedAt":"2026-11-10T17:34:11.199Z","items":[],"note":"Media was declared by the source, but no asset passed the supported availability checks. Open the original source to view it."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How would HOPE’s results change if it accounted for cross-layer interactions or interactions among three or more experts?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Your Experts Have Partners. There's HOPE for Keeping Them Together","summary":"Amazon author Alex M. Tseng presents HOPE, a method for pruning experts from Mixture-of-Experts (MoE) language models. Unlike first-order methods, which score experts independently, HOPE accounts for pairwise interactions: experts that often activate together may form cooperative groups, so removing one can impair the contribution of another. The authors say this matters most when pruning aggressively, because more cooperative links may be broken.\n\nThe post reports evaluations on three models, two calibration datasets and several benchmarks. Across the tested conditions, HOPE achieved the best average rank among five methods; at 50% pruning, its average rank was 1.58 versus 2.42 for the next-best method. On Qwen3.5-35B-A3B at that pruning rate, the authors report a score of 42.3 on SWE-Bench Pro, compared with 41.8 for the unpruned model and 36.2 for REAP.\n\nThe authors describe the gains as most pronounced at 40–50% pruning, while first-order methods are sometimes competitive at 10–20%. HOPE accounts for pairwise interactions within each layer, not cross-layer interactions; the authors say their cross-layer results remain preliminary. How would HOPE’s results change if it accounted for cross-layer interactions or interactions among three or more experts?","question":"How would HOPE’s results change if it accounted for cross-layer interactions or interactions among three or more experts?","claims":[{"text":"Alex M. Tseng presents HOPE, a pruning method that accounts for pairwise expert interactions in Mixture-of-Experts models.","excerpt":"Our newest paper derives HOPE (Higher-Order Pruning of Experts), a second-order method that accounts for pairwise expert interactions.","support":"source-stated"},{"text":"The authors report HOPE achieved the best average rank across the tested models, calibration sets and benchmarks.","excerpt":"Across all conditions, HOPE achieved the best average rank (2.07 out of 5 methods)","support":"source-stated"},{"text":"At 50% pruning, the authors report HOPE's average rank was 1.58, compared with 2.42 for REAP.","excerpt":"At 50% pruning, HOPE's average rank improves to 1.58 (vs. 2.42 for the next-best method, REAP).","support":"source-stated"},{"text":"On Qwen3.5-35B-A3B at 50% pruning, the authors report HOPE scored 42.3 on SWE-Bench Pro, versus 41.8 for the unpruned model and 36.2 for REAP.","excerpt":"On Qwen3.5-35B-A3B at 50% pruning, HOPE achieved 42.3 on SWE-Bench Pro, matching the unpruned model's 41.8 (while REAP dropped to 36.2, a 13% regression).","support":"source-stated"},{"text":"The authors say HOPE's advantage is most pronounced at 40–50% pruning; at 10–20%, first-order methods are sometimes competitive.","excerpt":"HOPE's advantage is most pronounced at higher pruning rates (40–50%). At lower pruning rates (10–20%), first-order methods like REAP and MAN are sometimes competitive","support":"source-stated"},{"text":"HOPE prunes layers independently and does not account for cross-layer interactions; the authors describe their cross-layer results as preliminary.","excerpt":"HOPE prunes each layer independently and does not account for cross-layer interactions. In our paper, we presented a theoretical framework for cross-layer HOPE, but the results remain preliminary.","support":"source-stated"},{"text":"HOPE captures pairwise interactions, while interactions among three or more experts could provide additional signal, according to the authors.","excerpt":"Finally, HOPE captures pairwise interactions, but even higher-order interactions (triplets and beyond) could provide additional signal","support":"source-stated"}],"limitations":["The reported results cover three models, two calibration datasets and the benchmarks described in the post; they do not establish performance beyond those evaluations.","The authors say HOPE's advantage shrinks at lower pruning rates, where first-order methods are sometimes competitive.","HOPE does not account for cross-layer interactions, and the authors describe their cross-layer results as preliminary.","HOPE models pairwise interactions; the post says interactions among three or more experts could provide additional signal but are difficult to estimate."],"mediaId":"55d059b88c6c576a2f3d9ab40518505078744d93ab6e7c4bc3d7b8ab49056416","correctionReason":null,"id":"ab90bf4c83956c6de48ff82f4b5aecde8912e50425298db52fe1bf38be68c4cd","publicationId":"744e4ab7-f45d-498a-8a6b-9461b7e96541","version":1,"sourceUrl":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","sourceTitle":"Your Experts Have Partners. There's HOPE for Keeping Them Together","publisher":"Hugging Face","publishedAt":"2026-11-09T12:34:29.000Z","originalPublished":{"value":"2026-11-09T12:34:29.000Z","precision":"instant","basis":"feed_entry"},"firstObservedAt":"2026-11-09T12:36:33.153Z","createdAt":"2026-11-10T17:03:59.661Z","historical":false,"entityIds":["hugging-face"],"topicIds":["intelligence-and-discovery","work-and-agency"],"media":[{"id":"55d059b88c6c576a2f3d9ab40518505078744d93ab6e7c4bc3d7b8ab49056416","kind":"article","url":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","sourceUrl":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T12:45:41.117Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"found","sourceUrl":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","checkedAt":"2026-11-10T17:34:11.199Z","items":[],"note":"Media was declared by the source, but no asset passed the supported availability checks. Open the original source to view it."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"55d059b88c6c576a2f3d9ab40518505078744d93ab6e7c4bc3d7b8ab49056416","confidence":0.99,"provider":"typesafe","model":"jev-1.13.0","inputHash":"72ab8b99ac0c43c82ea48ee74d941f87b706b6864f2b6b2b179fe4604e154f2f","decidedAt":"2026-11-09T12:45:41.996Z"},"provenance":{"textHash":"74d15c233a49ec339d9b1eee5a02633ec8c8406e33d7c881b3532315f4c67f61","observationId":"b65c78caee9e22b49fc9227c418eaf1caca12e369ca29818a159c67d4222911f","retrievedAt":"2026-11-09T12:45:40.946Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna; stream-draft-repair-v1","runId":"6301b998-8bc9-44d2-8092-3f134e1a9113"},"reviewer":{"model":"gpt-6-luna","runId":"c075cccc-9548-43f3-8a3e-250d9d27ad2d"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T17:03:59.661Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[],"authorRole":"article-author","hosts":[],"status":"missing","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[],"participationCoverage":{"total":0,"returned":0,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://huggingface.co/blog/amazon/higher-order-pruning-experts","snapshotId":"8fdd8a1213d4066302b5e5ceb9b41efdea93b248307559b3d71851afbf386bae","contentSha256":"551848b2c414602d347bf0070e451066c0446e11c7904ec2f93b52d968648b8e","textHash":"74d15c233a49ec339d9b1eee5a02633ec8c8406e33d7c881b3532315f4c67f61","observedAt":"2026-11-09T12:45:40.946Z","basis":"publisher-page-metadata","sourceLinks":42,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/ab90bf4c83956c6de48ff82f4b5aecde8912e50425298db52fe1bf38be68c4cd/source"}}}],"audits":[{"publicationId":"744e4ab7-f45d-498a-8a6b-9461b7e96541","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The post appears under Amazon on Hugging Face, with Alex M Tseng as author. The paper is multi-author, so presenting HOPE through Tseng and reporting results as 'the authors report' is consistent with the source.","excerpt":"Our newest paper derives HOPE (Higher-Order Pruning of Experts), a second-order method that accounts for pairwise expert interactions."},{"dimension":"summary","status":"uncertain","explanation":"The draft is faithful but does not say why expert pruning matters. It omits the post's central motivation: every expert must stay in memory even though only a few are used per token. It also leaves 'experts', 'pruning' (permanently deleting experts) and 'first-order' unexplained for non-specialists. Smallest repair: add one sentence on the memory bottleneck and what pruning does, plus term-library tooltips for MoE, expert, pruning, first-order and SWE-Bench Pro.","excerpt":"every expert MLP must occupy memory at inference time, even though the vast majority are idle for any given token"},{"dimension":"date","status":"pass","explanation":"originalPublished of 2026-11-09 matches the post's stated publication date. The arXiv paper may predate it, but the source under review is this blog post.","excerpt":"Published\nNovember 9, 2026"},{"dimension":"support","status":"pass","explanation":"All numeric claims, the conditions and the limitations match the source and are attributed. The draft's neutral 'compared with' avoids overstating the source's 'matching' claim. The closing question follows from the stated limitations and implies no unsupported benefit.","excerpt":"Finally, HOPE captures pairwise interactions, but even higher-order interactions (triplets and beyond) could provide additional signal, although they are combinatorially difficult to estimate."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the original Hugging Face post, the most relevant way to access the source.","excerpt":null}],"createdAt":"2026-11-10T17:03:59.661Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}