{"id":"467626767f2970c05870fff0e34568499393b0c0c9a4669b8936c35a678f2904","title":"On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning","argument":"Authors Yu Li, Yufa Zhou and Tian Lan argue that training-inference mismatch in LLM reinforcement learning has two sources: differences in engine numerics and stale weights during asynchronous sampling. Because the sources behave differently, a correction that works for one may fail for the other.\n\nThe authors describe numerical discrepancy as approximately constant, biasing updates by a nearly fixed amount. By contrast, policy lag arises from the trainer’s updates: the resulting bias enlarges the next lag, creating a feedback loop. They report that importance weighting cuts this loop only while the lag ratio stays concentrated; on Qwen3-1.7B, it fails when the ratio’s effective sample size falls below one half.\n\nTheir discrepancy-lag correction (DLC) centers numerical discrepancy, clips the lag ratio against sampler weights and resynchronizes the sampler when effective sample size falls below one half. In tests with Qwen3-0.6B and Qwen3-1.7B, the authors report that DLC kept the KL divergence from on-policy training within twice the KL between two independent on-policy trainings. They also report that resynchronization prevented collapses missed by per-token corrections and a KL measure on the sampler’s prefixes.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the abstract; it does not include detailed methods, experimental conditions, or results beyond the reported claims. The reported evaluations name Qwen3-0.6B and Qwen3-1.7B; the abstract does not establish performance on other models or conditions.","speaker":{"name":"Li, Yu, Zhou, Yufa, Lan, Tian","roleAtSource":"Article author"},"source":{"title":"On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.40945","publishedAt":"2026-10-29","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors identify engine numerics and stale weights in asynchronous sampling as two sources of training-inference mismatch, and say corrections suited to one source may fail for the other.","excerpt":"The mismatch has two sources, the numerics of the engine and the stale weights of asynchronous sampling, and a correction that stabilizes training under one source can fail under the other.","url":"https://arxiv.org/abs/2610.40945","relationship":"Source-stated"},{"text":"The authors characterize numerical discrepancy as approximately constant and policy lag as a feedback loop driven by the trainer’s own updates.","excerpt":"The numerical discrepancy stays approximately constant, so it biases the updates by a nearly fixed amount, and centering the score at each prefix removes its leading bias. The policy lag is produced by the trainer's own updates","url":"https://arxiv.org/abs/2610.40945","relationship":"Source-stated"},{"text":"The abstract reports that importance weighting fails on Qwen3-1.7B at the step when the lag ratio’s effective sample size falls below one half.","excerpt":"on Qwen3-1.7B it fails at the step where the effective sample size of the ratio falls below one half.","url":"https://arxiv.org/abs/2610.40945","relationship":"Source-stated"},{"text":"The authors report that DLC kept the measured KL divergence within twice the comparison between two independent on-policy trainings on Qwen3-0.6B and Qwen3-1.7B, and that resynchronization prevented collapses missed by other measures.","excerpt":"Under a stale sampler, DLC keeps the KL divergence from on-policy training within twice the KL between two independent on-policy trainings on Qwen3-0.6B and Qwen3-1.7B, and its resynchronization prevents collapses","url":"https://arxiv.org/abs/2610.40945","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.40945","checkedAt":"2026-11-11T04:32:08.428Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well does DLC perform beyond these two models and the conditions evaluated in the paper?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning","summary":"Authors Yu Li, Yufa Zhou and Tian Lan argue that training-inference mismatch in LLM reinforcement learning has two sources: differences in engine numerics and stale weights during asynchronous sampling. Because the sources behave differently, a correction that works for one may fail for the other.\n\nThe authors describe numerical discrepancy as approximately constant, biasing updates by a nearly fixed amount. By contrast, policy lag arises from the trainer’s updates: the resulting bias enlarges the next lag, creating a feedback loop. They report that importance weighting cuts this loop only while the lag ratio stays concentrated; on Qwen3-1.7B, it fails when the ratio’s effective sample size falls below one half.\n\nTheir discrepancy-lag correction (DLC) centers numerical discrepancy, clips the lag ratio against sampler weights and resynchronizes the sampler when effective sample size falls below one half. In tests with Qwen3-0.6B and Qwen3-1.7B, the authors report that DLC kept the KL divergence from on-policy training within twice the KL between two independent on-policy trainings. They also report that resynchronization prevented collapses missed by per-token corrections and a KL measure on the sampler’s prefixes.","question":"How well does DLC perform beyond these two models and the conditions evaluated in the paper?","claims":[{"text":"The authors identify engine numerics and stale weights in asynchronous sampling as two sources of training-inference mismatch, and say corrections suited to one source may fail for the other.","excerpt":"The mismatch has two sources, the numerics of the engine and the stale weights of asynchronous sampling, and a correction that stabilizes training under one source can fail under the other.","support":"source-stated"},{"text":"The authors characterize numerical discrepancy as approximately constant and policy lag as a feedback loop driven by the trainer’s own updates.","excerpt":"The numerical discrepancy stays approximately constant, so it biases the updates by a nearly fixed amount, and centering the score at each prefix removes its leading bias. The policy lag is produced by the trainer's own updates","support":"source-stated"},{"text":"The abstract reports that importance weighting fails on Qwen3-1.7B at the step when the lag ratio’s effective sample size falls below one half.","excerpt":"on Qwen3-1.7B it fails at the step where the effective sample size of the ratio falls below one half.","support":"source-stated"},{"text":"The authors report that DLC kept the measured KL divergence within twice the comparison between two independent on-policy trainings on Qwen3-0.6B and Qwen3-1.7B, and that resynchronization prevented collapses missed by other measures.","excerpt":"Under a stale sampler, DLC keeps the KL divergence from on-policy training within twice the KL between two independent on-policy trainings on Qwen3-0.6B and Qwen3-1.7B, and its resynchronization prevents collapses","support":"source-stated"}],"limitations":["The supplied evidence is the abstract; it does not include detailed methods, experimental conditions, or results beyond the reported claims.","The reported evaluations name Qwen3-0.6B and Qwen3-1.7B; the abstract does not establish performance on other models or conditions."],"mediaId":"1549edabf6f2bea6e59c2635b1375d61f1e4ba323d262e8a2a87e26ad0f56c16","correctionReason":null,"id":"467626767f2970c05870fff0e34568499393b0c0c9a4669b8936c35a678f2904","publicationId":"1b3d1c2b-aee1-4dd1-8419-6871f1ef1502","version":1,"sourceUrl":"https://arxiv.org/abs/2610.40945","sourceTitle":"On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-29","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-02T05:03:16.864Z","createdAt":"2026-11-11T02:25:55.827Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"1549edabf6f2bea6e59c2635b1375d61f1e4ba323d262e8a2a87e26ad0f56c16","kind":"article","url":"https://arxiv.org/abs/2610.40945","sourceUrl":"https://arxiv.org/abs/2610.40945","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-03T04:16:54.085Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.40945","checkedAt":"2026-11-11T04:32:08.428Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"1549edabf6f2bea6e59c2635b1375d61f1e4ba323d262e8a2a87e26ad0f56c16","confidence":0.8400000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"74e9603eb594da0026ec15ac4bae0120cf579f78a4dc8f8ec247a2416d86fd0b","decidedAt":"2026-11-03T04:16:56.251Z"},"provenance":{"textHash":"496f070b09cfabbde06a6efe0c7a677e9e233a40e0b1ed51ada3bbdd868995ab","observationId":"5ecb0aab8046e02dbc9c230215e7e728fd064d7f7ea7ff87a6088420b23af96b","retrievedAt":"2026-11-03T04:16:53.795Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"4449e6cf-a3da-4071-85c2-741f55909fe9"},"reviewer":{"model":"gpt-6-luna","runId":"7efb7f55-6560-48d4-8923-b874bfb15222"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-11T02:25:55.827Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Li, Yu","url":null,"kind":"unknown"},{"name":"Zhou, Yufa","url":null,"kind":"unknown"},{"name":"Lan, Tian","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Li, Yu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Li, Yu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.40945","locator":"head/meta[name=author|citation_author]"},"assertionId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T04:32:08.428Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef","sourceUrl":"https://arxiv.org/abs/2610.40945","contentSha256":"8c21a0da180cf1f8e98f0cf9188c412df0ae7496e6d25098b0e6f698ccb2c700","textHash":"496f070b09cfabbde06a6efe0c7a677e9e233a40e0b1ed51ada3bbdd868995ab","observedAt":"2026-11-11T04:32:08.428Z"}},{"actor":{"name":"Zhou, Yufa","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhou, Yufa","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.40945","locator":"head/meta[name=author|citation_author]"},"assertionId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T04:32:08.428Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef","sourceUrl":"https://arxiv.org/abs/2610.40945","contentSha256":"8c21a0da180cf1f8e98f0cf9188c412df0ae7496e6d25098b0e6f698ccb2c700","textHash":"496f070b09cfabbde06a6efe0c7a677e9e233a40e0b1ed51ada3bbdd868995ab","observedAt":"2026-11-11T04:32:08.428Z"}},{"actor":{"name":"Lan, Tian","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Lan, Tian","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.40945","locator":"head/meta[name=author|citation_author]"},"assertionId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T04:32:08.428Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef","sourceUrl":"https://arxiv.org/abs/2610.40945","contentSha256":"8c21a0da180cf1f8e98f0cf9188c412df0ae7496e6d25098b0e6f698ccb2c700","textHash":"496f070b09cfabbde06a6efe0c7a677e9e233a40e0b1ed51ada3bbdd868995ab","observedAt":"2026-11-11T04:32:08.428Z"}}],"participationCoverage":{"total":3,"returned":3,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.40945","snapshotId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef","contentSha256":"8c21a0da180cf1f8e98f0cf9188c412df0ae7496e6d25098b0e6f698ccb2c700","textHash":"496f070b09cfabbde06a6efe0c7a677e9e233a40e0b1ed51ada3bbdd868995ab","observedAt":"2026-11-11T04:32:08.428Z","basis":"publisher-page-metadata","sourceLinks":43,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/467626767f2970c05870fff0e34568499393b0c0c9a4669b8936c35a678f2904/source"}},"canonicalUrl":"https://escapemeridian.com/readings/on-the-origin-and-evolution-of-training-inference-mismatch-in-46762676","formats":{"html":"/readings/on-the-origin-and-evolution-of-training-inference-mismatch-in-46762676","markdown":"/readings/on-the-origin-and-evolution-of-training-inference-mismatch-in-46762676.md","json":"/readings/on-the-origin-and-evolution-of-training-inference-mismatch-in-46762676.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"},{"id":"term-reinforcement-learning","name":"Reinforcement learning","explanation":"Training by trial and error, in which a system is rewarded for the outcomes its designers want.","referenceUrl":"https://developers.google.com/machine-learning/glossary#reinforcement-learning-rl"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"467626767f2970c05870fff0e34568499393b0c0c9a4669b8936c35a678f2904","revision":"1b3d1c2b-aee1-4dd1-8419-6871f1ef1502","revisionKind":"publication-id","title":"On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning","url":"https://escapemeridian.com/readings/on-the-origin-and-evolution-of-training-inference-mismatch-in-46762676"},"source":{"type":"source","id":"https://arxiv.org/abs/2610.40945","revision":"496f070b09cfabbde06a6efe0c7a677e9e233a40e0b1ed51ada3bbdd868995ab","revisionKind":"sha256-source-text","title":"On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning","url":"https://arxiv.org/abs/2610.40945"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"467626767f2970c05870fff0e34568499393b0c0c9a4669b8936c35a678f2904","title":"On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning","argument":"Authors Yu Li, Yufa Zhou and Tian Lan argue that training-inference mismatch in LLM reinforcement learning has two sources: differences in engine numerics and stale weights during asynchronous sampling. Because the sources behave differently, a correction that works for one may fail for the other.\n\nThe authors describe numerical discrepancy as approximately constant, biasing updates by a nearly fixed amount. By contrast, policy lag arises from the trainer’s updates: the resulting bias enlarges the next lag, creating a feedback loop. They report that importance weighting cuts this loop only while the lag ratio stays concentrated; on Qwen3-1.7B, it fails when the ratio’s effective sample size falls below one half.\n\nTheir discrepancy-lag correction (DLC) centers numerical discrepancy, clips the lag ratio against sampler weights and resynchronizes the sampler when effective sample size falls below one half. In tests with Qwen3-0.6B and Qwen3-1.7B, the authors report that DLC kept the KL divergence from on-policy training within twice the KL between two independent on-policy trainings. They also report that resynchronization prevented collapses missed by per-token corrections and a KL measure on the sampler’s prefixes.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the abstract; it does not include detailed methods, experimental conditions, or results beyond the reported claims. The reported evaluations name Qwen3-0.6B and Qwen3-1.7B; the abstract does not establish performance on other models or conditions.","speaker":{"name":"Li, Yu, Zhou, Yufa, Lan, Tian","roleAtSource":"Article author"},"source":{"title":"On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.40945","publishedAt":"2026-10-29","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors identify engine numerics and stale weights in asynchronous sampling as two sources of training-inference mismatch, and say corrections suited to one source may fail for the other.","excerpt":"The mismatch has two sources, the numerics of the engine and the stale weights of asynchronous sampling, and a correction that stabilizes training under one source can fail under the other.","url":"https://arxiv.org/abs/2610.40945","relationship":"Source-stated"},{"text":"The authors characterize numerical discrepancy as approximately constant and policy lag as a feedback loop driven by the trainer’s own updates.","excerpt":"The numerical discrepancy stays approximately constant, so it biases the updates by a nearly fixed amount, and centering the score at each prefix removes its leading bias. The policy lag is produced by the trainer's own updates","url":"https://arxiv.org/abs/2610.40945","relationship":"Source-stated"},{"text":"The abstract reports that importance weighting fails on Qwen3-1.7B at the step when the lag ratio’s effective sample size falls below one half.","excerpt":"on Qwen3-1.7B it fails at the step where the effective sample size of the ratio falls below one half.","url":"https://arxiv.org/abs/2610.40945","relationship":"Source-stated"},{"text":"The authors report that DLC kept the measured KL divergence within twice the comparison between two independent on-policy trainings on Qwen3-0.6B and Qwen3-1.7B, and that resynchronization prevented collapses missed by other measures.","excerpt":"Under a stale sampler, DLC keeps the KL divergence from on-policy training within twice the KL between two independent on-policy trainings on Qwen3-0.6B and Qwen3-1.7B, and its resynchronization prevents collapses","url":"https://arxiv.org/abs/2610.40945","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.40945","checkedAt":"2026-11-11T04:32:08.428Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well does DLC perform beyond these two models and the conditions evaluated in the paper?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning","summary":"Authors Yu Li, Yufa Zhou and Tian Lan argue that training-inference mismatch in LLM reinforcement learning has two sources: differences in engine numerics and stale weights during asynchronous sampling. Because the sources behave differently, a correction that works for one may fail for the other.\n\nThe authors describe numerical discrepancy as approximately constant, biasing updates by a nearly fixed amount. By contrast, policy lag arises from the trainer’s updates: the resulting bias enlarges the next lag, creating a feedback loop. They report that importance weighting cuts this loop only while the lag ratio stays concentrated; on Qwen3-1.7B, it fails when the ratio’s effective sample size falls below one half.\n\nTheir discrepancy-lag correction (DLC) centers numerical discrepancy, clips the lag ratio against sampler weights and resynchronizes the sampler when effective sample size falls below one half. In tests with Qwen3-0.6B and Qwen3-1.7B, the authors report that DLC kept the KL divergence from on-policy training within twice the KL between two independent on-policy trainings. They also report that resynchronization prevented collapses missed by per-token corrections and a KL measure on the sampler’s prefixes.","question":"How well does DLC perform beyond these two models and the conditions evaluated in the paper?","claims":[{"text":"The authors identify engine numerics and stale weights in asynchronous sampling as two sources of training-inference mismatch, and say corrections suited to one source may fail for the other.","excerpt":"The mismatch has two sources, the numerics of the engine and the stale weights of asynchronous sampling, and a correction that stabilizes training under one source can fail under the other.","support":"source-stated"},{"text":"The authors characterize numerical discrepancy as approximately constant and policy lag as a feedback loop driven by the trainer’s own updates.","excerpt":"The numerical discrepancy stays approximately constant, so it biases the updates by a nearly fixed amount, and centering the score at each prefix removes its leading bias. The policy lag is produced by the trainer's own updates","support":"source-stated"},{"text":"The abstract reports that importance weighting fails on Qwen3-1.7B at the step when the lag ratio’s effective sample size falls below one half.","excerpt":"on Qwen3-1.7B it fails at the step where the effective sample size of the ratio falls below one half.","support":"source-stated"},{"text":"The authors report that DLC kept the measured KL divergence within twice the comparison between two independent on-policy trainings on Qwen3-0.6B and Qwen3-1.7B, and that resynchronization prevented collapses missed by other measures.","excerpt":"Under a stale sampler, DLC keeps the KL divergence from on-policy training within twice the KL between two independent on-policy trainings on Qwen3-0.6B and Qwen3-1.7B, and its resynchronization prevents collapses","support":"source-stated"}],"limitations":["The supplied evidence is the abstract; it does not include detailed methods, experimental conditions, or results beyond the reported claims.","The reported evaluations name Qwen3-0.6B and Qwen3-1.7B; the abstract does not establish performance on other models or conditions."],"mediaId":"1549edabf6f2bea6e59c2635b1375d61f1e4ba323d262e8a2a87e26ad0f56c16","correctionReason":null,"id":"467626767f2970c05870fff0e34568499393b0c0c9a4669b8936c35a678f2904","publicationId":"1b3d1c2b-aee1-4dd1-8419-6871f1ef1502","version":1,"sourceUrl":"https://arxiv.org/abs/2610.40945","sourceTitle":"On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-29","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-02T05:03:16.864Z","createdAt":"2026-11-11T02:25:55.827Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"1549edabf6f2bea6e59c2635b1375d61f1e4ba323d262e8a2a87e26ad0f56c16","kind":"article","url":"https://arxiv.org/abs/2610.40945","sourceUrl":"https://arxiv.org/abs/2610.40945","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-03T04:16:54.085Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.40945","checkedAt":"2026-11-11T04:32:08.428Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"1549edabf6f2bea6e59c2635b1375d61f1e4ba323d262e8a2a87e26ad0f56c16","confidence":0.8400000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"74e9603eb594da0026ec15ac4bae0120cf579f78a4dc8f8ec247a2416d86fd0b","decidedAt":"2026-11-03T04:16:56.251Z"},"provenance":{"textHash":"496f070b09cfabbde06a6efe0c7a677e9e233a40e0b1ed51ada3bbdd868995ab","observationId":"5ecb0aab8046e02dbc9c230215e7e728fd064d7f7ea7ff87a6088420b23af96b","retrievedAt":"2026-11-03T04:16:53.795Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"4449e6cf-a3da-4071-85c2-741f55909fe9"},"reviewer":{"model":"gpt-6-luna","runId":"7efb7f55-6560-48d4-8923-b874bfb15222"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-11T02:25:55.827Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Li, Yu","url":null,"kind":"unknown"},{"name":"Zhou, Yufa","url":null,"kind":"unknown"},{"name":"Lan, Tian","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Li, Yu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Li, Yu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.40945","locator":"head/meta[name=author|citation_author]"},"assertionId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T04:32:08.428Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef","sourceUrl":"https://arxiv.org/abs/2610.40945","contentSha256":"8c21a0da180cf1f8e98f0cf9188c412df0ae7496e6d25098b0e6f698ccb2c700","textHash":"496f070b09cfabbde06a6efe0c7a677e9e233a40e0b1ed51ada3bbdd868995ab","observedAt":"2026-11-11T04:32:08.428Z"}},{"actor":{"name":"Zhou, Yufa","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhou, Yufa","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.40945","locator":"head/meta[name=author|citation_author]"},"assertionId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T04:32:08.428Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef","sourceUrl":"https://arxiv.org/abs/2610.40945","contentSha256":"8c21a0da180cf1f8e98f0cf9188c412df0ae7496e6d25098b0e6f698ccb2c700","textHash":"496f070b09cfabbde06a6efe0c7a677e9e233a40e0b1ed51ada3bbdd868995ab","observedAt":"2026-11-11T04:32:08.428Z"}},{"actor":{"name":"Lan, Tian","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Lan, Tian","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.40945","locator":"head/meta[name=author|citation_author]"},"assertionId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T04:32:08.428Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef","sourceUrl":"https://arxiv.org/abs/2610.40945","contentSha256":"8c21a0da180cf1f8e98f0cf9188c412df0ae7496e6d25098b0e6f698ccb2c700","textHash":"496f070b09cfabbde06a6efe0c7a677e9e233a40e0b1ed51ada3bbdd868995ab","observedAt":"2026-11-11T04:32:08.428Z"}}],"participationCoverage":{"total":3,"returned":3,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.40945","snapshotId":"14df2123be01045b247494b68243d69df8130decdc497b21c32c337ed567dfef","contentSha256":"8c21a0da180cf1f8e98f0cf9188c412df0ae7496e6d25098b0e6f698ccb2c700","textHash":"496f070b09cfabbde06a6efe0c7a677e9e233a40e0b1ed51ada3bbdd868995ab","observedAt":"2026-11-11T04:32:08.428Z","basis":"publisher-page-metadata","sourceLinks":43,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/467626767f2970c05870fff0e34568499393b0c0c9a4669b8936c35a678f2904/source"}}}],"audits":[{"publicationId":"1b3d1c2b-aee1-4dd1-8419-6871f1ef1502","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading correctly attributes the work to Yu Li, Yufa Zhou and Tian Lan. It frames the mechanisms and results as the authors' descriptions and reports rather than as established facts.","excerpt":"Authors:Yu Li, Yufa Zhou, Tian Lan"},{"dimension":"summary","status":"uncertain","explanation":"The summary never explains what training-inference mismatch is. The abstract says an inference engine samples responses while a separate trainer computes gradients, which makes the training data off-policy. Without that, a general reader cannot follow the central problem. On-policy, importance weighting, effective sample size and KL divergence are also left unexplained; they need tooltips. The reading also does not say why the work matters, namely training stability and collapse prevention. Smallest repair: add one plain sentence on the engine/trainer split and why off-policy data destabilizes training.","excerpt":"In practice, an inference engine samples the responses and a separate trainer computes the gradients, and the \\textbf{training-inference mismatch} between the two makes the training data off-policy."},{"dimension":"date","status":"pass","explanation":"originalPublished 2026-10-29 matches the v1 submission date on arXiv.","excerpt":"[Submitted on 29 Oct 2026]"},{"dimension":"support","status":"uncertain","explanation":"Most claims track the abstract closely. However, the summary paragraph and claim 4 drop the condition that the DLC KL result was obtained under a stale sampler. That condition scopes the result. Smallest repair: insert 'Under a stale sampler,' before the DLC KL result. The question about generalization beyond the two models follows reasonably from the evidence.","excerpt":"Under a stale sampler, DLC keeps the KL divergence from on-policy training within twice the KL between two independent on-policy trainings on Qwen3-0.6B and Qwen3-1.7B"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper, the most direct access to the source.","excerpt":null}],"createdAt":"2026-11-11T02:25:55.827Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}