{"id":"856da81b07ffff8bf3a9178f03e837b7bf1b08c37a36ce2f3313727765ab4a2f","title":"arXiv cs.AI: Proposing Better Rollouts for On-Policy Distillation","argument":"In “Proposing Better Rollouts for On-Policy Distillation,” Sirui Chen and coauthors argue that on-policy distillation can go wrong when the student’s own sampled trajectories are used as rollouts in every training round. They say the teacher may give noisy or misleading supervision on trajectories that are less reachable by the teacher, encouraging patterns such as repetition that can persist or grow in later rounds.\n\nThe authors propose Mixture Proposal On-Policy Distillation (MPOPD), which combines the student and teacher policies into a rollout distribution and approximates it with a proposal policy. They report consistent gains over on-policy distillation and recent variants on 16 benchmarks. On math reasoning, they report a 6.91 improvement in average accuracy over OPD for Qwen3-4B-Base, alongside a 41% reduction in average generation length. These are reported benchmark results; the abstract does not establish how well the method transfers to other tasks or settings.\n\nAn open question is whether MPOPD’s reported gains hold beyond the benchmarks and model setting described in the abstract.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evidence is the abstract; it does not provide benchmark-level results, experimental details, or evidence about performance outside the reported evaluations. The reported results do not establish how well MPOPD transfers to other tasks or model settings.","speaker":{"name":"Chen, Sirui, Qi, Yunzhe, Xu, Haobo, Zou, Jiaru, Ai, Mengting, Li, Zihao, He, Jingrui","roleAtSource":"Article author"},"source":{"title":"Proposing Better Rollouts for On-Policy Distillation","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.36477","publishedAt":"2026-10-27","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors argue that fixed student-generated rollouts can lead to noisy or misleading teacher supervision, which may promote repetition and persist across training rounds.","excerpt":"On such trajectories, teacher supervision can become noisy or even misleading, leading to undesirable student patterns such as repetition.","url":"https://arxiv.org/abs/2610.36477","relationship":"Source-stated"},{"text":"The authors propose MPOPD, which uses a rollout distribution combining student and teacher policies and a proposal policy to approximate it.","excerpt":"The resulting target is a geometric mixture of the student and teacher policies, and we introduce a proposal policy that approximates this mixture","url":"https://arxiv.org/abs/2610.36477","relationship":"Source-stated"},{"text":"The abstract reports consistent gains over OPD and recent variants across 16 math, knowledge, and medical reasoning benchmarks.","excerpt":"Experiments on 16 math, knowledge, and medical reasoning benchmarks show consistent gains over OPD and recent variants.","url":"https://arxiv.org/abs/2610.36477","relationship":"Source-stated"},{"text":"For Qwen3-4B-Base on math reasoning, the authors report a 6.91 average-accuracy improvement over OPD and a 41% reduction in average generation length.","excerpt":"improves average accuracy by 6.91 over OPD on Qwen3-4B-Base while reducing average generation length by 41%","url":"https://arxiv.org/abs/2610.36477","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36477","checkedAt":"2026-11-14T03:20:08.648Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Do MPOPD’s reported gains hold on tasks and model settings beyond those evaluated in the paper?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Proposing Better Rollouts for On-Policy Distillation","summary":"In “Proposing Better Rollouts for On-Policy Distillation,” Sirui Chen and coauthors argue that on-policy distillation can go wrong when the student’s own sampled trajectories are used as rollouts in every training round. They say the teacher may give noisy or misleading supervision on trajectories that are less reachable by the teacher, encouraging patterns such as repetition that can persist or grow in later rounds.\n\nThe authors propose Mixture Proposal On-Policy Distillation (MPOPD), which combines the student and teacher policies into a rollout distribution and approximates it with a proposal policy. They report consistent gains over on-policy distillation and recent variants on 16 benchmarks. On math reasoning, they report a 6.91 improvement in average accuracy over OPD for Qwen3-4B-Base, alongside a 41% reduction in average generation length. These are reported benchmark results; the abstract does not establish how well the method transfers to other tasks or settings.\n\nAn open question is whether MPOPD’s reported gains hold beyond the benchmarks and model setting described in the abstract.","question":"Do MPOPD’s reported gains hold on tasks and model settings beyond those evaluated in the paper?","claims":[{"text":"The authors argue that fixed student-generated rollouts can lead to noisy or misleading teacher supervision, which may promote repetition and persist across training rounds.","excerpt":"On such trajectories, teacher supervision can become noisy or even misleading, leading to undesirable student patterns such as repetition.","support":"source-stated"},{"text":"The authors propose MPOPD, which uses a rollout distribution combining student and teacher policies and a proposal policy to approximate it.","excerpt":"The resulting target is a geometric mixture of the student and teacher policies, and we introduce a proposal policy that approximates this mixture","support":"source-stated"},{"text":"The abstract reports consistent gains over OPD and recent variants across 16 math, knowledge, and medical reasoning benchmarks.","excerpt":"Experiments on 16 math, knowledge, and medical reasoning benchmarks show consistent gains over OPD and recent variants.","support":"source-stated"},{"text":"For Qwen3-4B-Base on math reasoning, the authors report a 6.91 average-accuracy improvement over OPD and a 41% reduction in average generation length.","excerpt":"improves average accuracy by 6.91 over OPD on Qwen3-4B-Base while reducing average generation length by 41%","support":"source-stated"}],"limitations":["The evidence is the abstract; it does not provide benchmark-level results, experimental details, or evidence about performance outside the reported evaluations.","The reported results do not establish how well MPOPD transfers to other tasks or model settings."],"mediaId":"898b66c147311eec43cf3c487dcd43a1b942fd6e24a68b8bf609d7d6cd16fae1","correctionReason":null,"id":"856da81b07ffff8bf3a9178f03e837b7bf1b08c37a36ce2f3313727765ab4a2f","publicationId":"520d9fd4-4a40-41f5-82ec-2298b9ad082a","version":1,"sourceUrl":"https://arxiv.org/abs/2610.36477","sourceTitle":"Proposing Better Rollouts for On-Policy Distillation","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-27","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-28T04:48:47.197Z","createdAt":"2026-11-14T02:49:10.516Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"898b66c147311eec43cf3c487dcd43a1b942fd6e24a68b8bf609d7d6cd16fae1","kind":"article","url":"https://arxiv.org/abs/2610.36477","sourceUrl":"https://arxiv.org/abs/2610.36477","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-30T03:04:23.420Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36477","checkedAt":"2026-11-14T03:20:08.648Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"898b66c147311eec43cf3c487dcd43a1b942fd6e24a68b8bf609d7d6cd16fae1","confidence":0.86,"provider":"typesafe","model":"jev-1.13.0","inputHash":"4caaff82bbdf15e617b068280a3e2c3a07d1fcc131e261313d59d9e4f7cde3f7","decidedAt":"2026-10-30T03:04:24.900Z"},"provenance":{"textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observationId":"ef73142f9ab2606273072c5c3ed4a1e1036adaa59044057cbce7e5badee7d428","retrievedAt":"2026-10-30T03:04:23.256Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"6615aa86-18e6-44c1-857d-f4d6194ef3d1"},"reviewer":{"model":"gpt-6-luna","runId":"9aa6e54d-7c30-4aee-887f-a908be31bd49"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-14T02:49:10.516Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Chen, Sirui","url":null,"kind":"unknown"},{"name":"Qi, Yunzhe","url":null,"kind":"unknown"},{"name":"Xu, Haobo","url":null,"kind":"unknown"},{"name":"Zou, Jiaru","url":null,"kind":"unknown"},{"name":"Ai, Mengting","url":null,"kind":"unknown"},{"name":"Li, Zihao","url":null,"kind":"unknown"},{"name":"He, Jingrui","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Chen, Sirui","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chen, Sirui","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}},{"actor":{"name":"Qi, Yunzhe","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Qi, Yunzhe","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}},{"actor":{"name":"Xu, Haobo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Xu, Haobo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}},{"actor":{"name":"Zou, Jiaru","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zou, Jiaru","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}},{"actor":{"name":"Ai, Mengting","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ai, Mengting","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}},{"actor":{"name":"Li, Zihao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Li, Zihao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}},{"actor":{"name":"He, Jingrui","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"He, Jingrui","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}}],"participationCoverage":{"total":7,"returned":7,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.36477","snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z","basis":"publisher-page-metadata","sourceLinks":49,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/856da81b07ffff8bf3a9178f03e837b7bf1b08c37a36ce2f3313727765ab4a2f/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-proposing-better-rollouts-for-on-policy-distillation-856da81b","formats":{"html":"/readings/arxiv-cs-ai-proposing-better-rollouts-for-on-policy-distillation-856da81b","markdown":"/readings/arxiv-cs-ai-proposing-better-rollouts-for-on-policy-distillation-856da81b.md","json":"/readings/arxiv-cs-ai-proposing-better-rollouts-for-on-policy-distillation-856da81b.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"856da81b07ffff8bf3a9178f03e837b7bf1b08c37a36ce2f3313727765ab4a2f","revision":"520d9fd4-4a40-41f5-82ec-2298b9ad082a","revisionKind":"publication-id","title":"arXiv cs.AI: Proposing Better Rollouts for On-Policy Distillation","url":"https://escapemeridian.com/readings/arxiv-cs-ai-proposing-better-rollouts-for-on-policy-distillation-856da81b"},"source":{"type":"source","id":"https://arxiv.org/abs/2610.36477","revision":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","revisionKind":"sha256-source-text","title":"Proposing Better Rollouts for On-Policy Distillation","url":"https://arxiv.org/abs/2610.36477"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"856da81b07ffff8bf3a9178f03e837b7bf1b08c37a36ce2f3313727765ab4a2f","title":"arXiv cs.AI: Proposing Better Rollouts for On-Policy Distillation","argument":"In “Proposing Better Rollouts for On-Policy Distillation,” Sirui Chen and coauthors argue that on-policy distillation can go wrong when the student’s own sampled trajectories are used as rollouts in every training round. They say the teacher may give noisy or misleading supervision on trajectories that are less reachable by the teacher, encouraging patterns such as repetition that can persist or grow in later rounds.\n\nThe authors propose Mixture Proposal On-Policy Distillation (MPOPD), which combines the student and teacher policies into a rollout distribution and approximates it with a proposal policy. They report consistent gains over on-policy distillation and recent variants on 16 benchmarks. On math reasoning, they report a 6.91 improvement in average accuracy over OPD for Qwen3-4B-Base, alongside a 41% reduction in average generation length. These are reported benchmark results; the abstract does not establish how well the method transfers to other tasks or settings.\n\nAn open question is whether MPOPD’s reported gains hold beyond the benchmarks and model setting described in the abstract.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evidence is the abstract; it does not provide benchmark-level results, experimental details, or evidence about performance outside the reported evaluations. The reported results do not establish how well MPOPD transfers to other tasks or model settings.","speaker":{"name":"Chen, Sirui, Qi, Yunzhe, Xu, Haobo, Zou, Jiaru, Ai, Mengting, Li, Zihao, He, Jingrui","roleAtSource":"Article author"},"source":{"title":"Proposing Better Rollouts for On-Policy Distillation","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.36477","publishedAt":"2026-10-27","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors argue that fixed student-generated rollouts can lead to noisy or misleading teacher supervision, which may promote repetition and persist across training rounds.","excerpt":"On such trajectories, teacher supervision can become noisy or even misleading, leading to undesirable student patterns such as repetition.","url":"https://arxiv.org/abs/2610.36477","relationship":"Source-stated"},{"text":"The authors propose MPOPD, which uses a rollout distribution combining student and teacher policies and a proposal policy to approximate it.","excerpt":"The resulting target is a geometric mixture of the student and teacher policies, and we introduce a proposal policy that approximates this mixture","url":"https://arxiv.org/abs/2610.36477","relationship":"Source-stated"},{"text":"The abstract reports consistent gains over OPD and recent variants across 16 math, knowledge, and medical reasoning benchmarks.","excerpt":"Experiments on 16 math, knowledge, and medical reasoning benchmarks show consistent gains over OPD and recent variants.","url":"https://arxiv.org/abs/2610.36477","relationship":"Source-stated"},{"text":"For Qwen3-4B-Base on math reasoning, the authors report a 6.91 average-accuracy improvement over OPD and a 41% reduction in average generation length.","excerpt":"improves average accuracy by 6.91 over OPD on Qwen3-4B-Base while reducing average generation length by 41%","url":"https://arxiv.org/abs/2610.36477","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36477","checkedAt":"2026-11-14T03:20:08.648Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Do MPOPD’s reported gains hold on tasks and model settings beyond those evaluated in the paper?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Proposing Better Rollouts for On-Policy Distillation","summary":"In “Proposing Better Rollouts for On-Policy Distillation,” Sirui Chen and coauthors argue that on-policy distillation can go wrong when the student’s own sampled trajectories are used as rollouts in every training round. They say the teacher may give noisy or misleading supervision on trajectories that are less reachable by the teacher, encouraging patterns such as repetition that can persist or grow in later rounds.\n\nThe authors propose Mixture Proposal On-Policy Distillation (MPOPD), which combines the student and teacher policies into a rollout distribution and approximates it with a proposal policy. They report consistent gains over on-policy distillation and recent variants on 16 benchmarks. On math reasoning, they report a 6.91 improvement in average accuracy over OPD for Qwen3-4B-Base, alongside a 41% reduction in average generation length. These are reported benchmark results; the abstract does not establish how well the method transfers to other tasks or settings.\n\nAn open question is whether MPOPD’s reported gains hold beyond the benchmarks and model setting described in the abstract.","question":"Do MPOPD’s reported gains hold on tasks and model settings beyond those evaluated in the paper?","claims":[{"text":"The authors argue that fixed student-generated rollouts can lead to noisy or misleading teacher supervision, which may promote repetition and persist across training rounds.","excerpt":"On such trajectories, teacher supervision can become noisy or even misleading, leading to undesirable student patterns such as repetition.","support":"source-stated"},{"text":"The authors propose MPOPD, which uses a rollout distribution combining student and teacher policies and a proposal policy to approximate it.","excerpt":"The resulting target is a geometric mixture of the student and teacher policies, and we introduce a proposal policy that approximates this mixture","support":"source-stated"},{"text":"The abstract reports consistent gains over OPD and recent variants across 16 math, knowledge, and medical reasoning benchmarks.","excerpt":"Experiments on 16 math, knowledge, and medical reasoning benchmarks show consistent gains over OPD and recent variants.","support":"source-stated"},{"text":"For Qwen3-4B-Base on math reasoning, the authors report a 6.91 average-accuracy improvement over OPD and a 41% reduction in average generation length.","excerpt":"improves average accuracy by 6.91 over OPD on Qwen3-4B-Base while reducing average generation length by 41%","support":"source-stated"}],"limitations":["The evidence is the abstract; it does not provide benchmark-level results, experimental details, or evidence about performance outside the reported evaluations.","The reported results do not establish how well MPOPD transfers to other tasks or model settings."],"mediaId":"898b66c147311eec43cf3c487dcd43a1b942fd6e24a68b8bf609d7d6cd16fae1","correctionReason":null,"id":"856da81b07ffff8bf3a9178f03e837b7bf1b08c37a36ce2f3313727765ab4a2f","publicationId":"520d9fd4-4a40-41f5-82ec-2298b9ad082a","version":1,"sourceUrl":"https://arxiv.org/abs/2610.36477","sourceTitle":"Proposing Better Rollouts for On-Policy Distillation","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-27","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-28T04:48:47.197Z","createdAt":"2026-11-14T02:49:10.516Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"898b66c147311eec43cf3c487dcd43a1b942fd6e24a68b8bf609d7d6cd16fae1","kind":"article","url":"https://arxiv.org/abs/2610.36477","sourceUrl":"https://arxiv.org/abs/2610.36477","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-30T03:04:23.420Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36477","checkedAt":"2026-11-14T03:20:08.648Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"898b66c147311eec43cf3c487dcd43a1b942fd6e24a68b8bf609d7d6cd16fae1","confidence":0.86,"provider":"typesafe","model":"jev-1.13.0","inputHash":"4caaff82bbdf15e617b068280a3e2c3a07d1fcc131e261313d59d9e4f7cde3f7","decidedAt":"2026-10-30T03:04:24.900Z"},"provenance":{"textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observationId":"ef73142f9ab2606273072c5c3ed4a1e1036adaa59044057cbce7e5badee7d428","retrievedAt":"2026-10-30T03:04:23.256Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"6615aa86-18e6-44c1-857d-f4d6194ef3d1"},"reviewer":{"model":"gpt-6-luna","runId":"9aa6e54d-7c30-4aee-887f-a908be31bd49"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-14T02:49:10.516Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Chen, Sirui","url":null,"kind":"unknown"},{"name":"Qi, Yunzhe","url":null,"kind":"unknown"},{"name":"Xu, Haobo","url":null,"kind":"unknown"},{"name":"Zou, Jiaru","url":null,"kind":"unknown"},{"name":"Ai, Mengting","url":null,"kind":"unknown"},{"name":"Li, Zihao","url":null,"kind":"unknown"},{"name":"He, Jingrui","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Chen, Sirui","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chen, Sirui","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}},{"actor":{"name":"Qi, Yunzhe","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Qi, Yunzhe","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}},{"actor":{"name":"Xu, Haobo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Xu, Haobo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}},{"actor":{"name":"Zou, Jiaru","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zou, Jiaru","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}},{"actor":{"name":"Ai, Mengting","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ai, Mengting","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}},{"actor":{"name":"Li, Zihao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Li, Zihao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}},{"actor":{"name":"He, Jingrui","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"He, Jingrui","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36477","locator":"head/meta[name=author|citation_author]"},"assertionId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-14T03:20:08.648Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","sourceUrl":"https://arxiv.org/abs/2610.36477","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z"}}],"participationCoverage":{"total":7,"returned":7,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.36477","snapshotId":"b9543eb74a8cac58df9005c224eb03c8044c35ee1271e1fd80da1c52fef8b813","contentSha256":"31c44da77f7ed52b1cfb338802435eb99ae01f1e883323ed62498a94e63ecd19","textHash":"6b62ac154332ee068ea526b60383bf6f20695d7388fa0307a3369cba14b6bebb","observedAt":"2026-11-14T03:20:08.648Z","basis":"publisher-page-metadata","sourceLinks":49,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/856da81b07ffff8bf3a9178f03e837b7bf1b08c37a36ce2f3313727765ab4a2f/source"}}}],"audits":[{"publicationId":"520d9fd4-4a40-41f5-82ec-2298b9ad082a","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The draft correctly attributes the paper to Sirui Chen and coauthors on arXiv. It keeps the arguments and benchmark results framed as the authors' reported claims.","excerpt":"Authors:Sirui Chen, Yunzhe Qi, Haobo Xu, Jiaru Zou, Mengting Ai, Zihao Li, Jingrui He"},{"dimension":"summary","status":"uncertain","explanation":"The summary is faithful, but a non-specialist could not follow it. It never explains on-policy distillation: a smaller student model learns from its own sampled outputs while a teacher model supervises. It also never says that OPD is a language-model post-training method or what rollouts, policy and trajectories mean. It omits the authors' central thesis that the rollout policy itself should be an optimization target, and the single-forward-pass point. The closing open-question sentence repeats the question. Smallest repair: add a one-sentence plain explanation of OPD and rollouts, and state the authors' thesis.","excerpt":"On-policy distillation (OPD) improves language-model post-training by letting the student learn from its own sampled trajectories under teacher supervision"},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-10-27, which matches the v1 submission date. It is not the date of the later v2 revision or the feed date.","excerpt":"Submitted on 27 Oct 2026 (v1), last revised 28 Oct 2026 (this version, v2)"},{"dimension":"support","status":"pass","explanation":"Every claim matches the abstract, including the mismatch mechanism, the geometric mixture, the 16 benchmarks, the 6.91 accuracy gain and the 41% length reduction. The limitations and the question are appropriately conditional and do not overstate real-world reliability.","excerpt":"On math reasoning, our method improves average accuracy by 6.91 over OPD on Qwen3-4B-Base while reducing average generation length by 41%"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified arXiv abstract page, which is the direct original source.","excerpt":null}],"createdAt":"2026-11-14T02:49:10.516Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}