{"id":"0fe488ea7cb42fdb7445e8431697520d90aef8b7260f52d9a9c61ef4338413ab","title":"arXiv cs.AI: Zeyuan Allen-Zhu says standard benchmarks miss model differences and weaknesses","argument":"In the arXiv abstract, Zeyuan Allen-Zhu reports that architectural differences can be hard to detect at academic-scale pretraining, but become measurable at larger scales. Canon layers add local token mixing and, the paper reports, improve Transformers while using about 30% fewer pretraining tokens for the same MMLU accuracy at 8B. With Canon, gated linear attention matches Gated DeltaNet and outperforms Mamba2 in the reported comparison. Allen-Zhu suggests that some architectural machinery may mainly mix neighboring tokens.\n\nThe paper also argues that standard benchmark scores can hide weaknesses in basic in-context skills. Linear models that match or beat Transformers on these benchmarks cannot reliably retrieve information from a short 500-token context. No tested model reliably beats random guessing on the paper’s two-hop question. Allen-Zhu says small synthetic playgrounds predicted architectural differences that standard benchmarks missed. He recommends including in-context reasoning examples in pretraining data early, rather than reserving them for later training stages. The findings challenge research based primarily on standard benchmark scores; the abstract reports experiments at particular scales and does not establish how broadly the results generalize.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract describes experiments at 1.3–8B parameters and 1–2T tokens; it does not establish that the reported results generalize beyond those tested scales and training setup. The retrieval and two-hop findings are described as reliability results in the paper’s tested tasks; they do not establish performance on all contexts or reasoning tasks. The supplied evidence is the arXiv abstract, not the full paper, so it does not provide the detailed experimental methods or task definitions.","speaker":{"name":"Allen-Zhu, Zeyuan","roleAtSource":"Article author"},"source":{"title":"Physics of Language Models: Part 4.2, Canon Layers at Scale where Synthetic Pretraining Resonates in Reality","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.24963","publishedAt":"2026-11-12","dateBasis":"Date basis not provided."},"evidence":[{"text":"The paper reports 64 from-scratch pretraining runs of Transformers and three linear-model families at 1.3–8B parameters, trained on 1–2T tokens.","excerpt":"We study it with 64 from-scratch pretraining runs of Transformers and three linear-model families at 1.3-8B parameters on 1-2T Nemotron-CC tokens","url":"https://arxiv.org/abs/2611.24963","relationship":"Source-stated"},{"text":"At 8B, Canon layers improved Transformers, requiring about 30% fewer pretraining tokens for the same MMLU accuracy.","excerpt":"Canon layers, which add local token mixing, consistently improve Transformers, saving about 30% of pretraining tokens for the same MMLU accuracy at 8B.","url":"https://arxiv.org/abs/2611.24963","relationship":"Source-stated"},{"text":"With Canon, gated linear attention matched Gated DeltaNet and outperformed Mamba2 in the reported 8B/1T comparison.","excerpt":"with Canon, the simplest linear model, gated linear attention, matches Gated DeltaNet and outperforms Mamba2 even at 8B/1T.","url":"https://arxiv.org/abs/2611.24963","relationship":"Source-stated"},{"text":"The paper reports that benchmark performance did not imply reliable retrieval from a short context or success on the tested two-hop question.","excerpt":"linear models that match or beat Transformers on these benchmarks cannot, in fact, reliably retrieve information from a short 500-token context. Moreover, no model, Transformer or linear, reliably beats random guessing on arguably the simplest two-hop question","url":"https://arxiv.org/abs/2611.24963","relationship":"Source-stated"},{"text":"Allen-Zhu recommends including in-context reasoning examples in pretraining data early, rather than reserving them for later training stages.","excerpt":"include in-context reasoning examples in pretraining data from an early stage, rather than reserve them for later mid-training, let alone post-training.","url":"https://arxiv.org/abs/2611.24963","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.24963","checkedAt":"2026-11-21T02:52:10.059Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Would including in-context reasoning examples in pretraining data early improve performance on the short-context retrieval and two-hop tasks described in the paper?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Zeyuan Allen-Zhu says standard benchmarks miss model differences and weaknesses","summary":"In the arXiv abstract, Zeyuan Allen-Zhu reports that architectural differences can be hard to detect at academic-scale pretraining, but become measurable at larger scales. Canon layers add local token mixing and, the paper reports, improve Transformers while using about 30% fewer pretraining tokens for the same MMLU accuracy at 8B. With Canon, gated linear attention matches Gated DeltaNet and outperforms Mamba2 in the reported comparison. Allen-Zhu suggests that some architectural machinery may mainly mix neighboring tokens.\n\nThe paper also argues that standard benchmark scores can hide weaknesses in basic in-context skills. Linear models that match or beat Transformers on these benchmarks cannot reliably retrieve information from a short 500-token context. No tested model reliably beats random guessing on the paper’s two-hop question. Allen-Zhu says small synthetic playgrounds predicted architectural differences that standard benchmarks missed. He recommends including in-context reasoning examples in pretraining data early, rather than reserving them for later training stages. The findings challenge research based primarily on standard benchmark scores; the abstract reports experiments at particular scales and does not establish how broadly the results generalize.","question":"Would including in-context reasoning examples in pretraining data early improve performance on the short-context retrieval and two-hop tasks described in the paper?","claims":[{"text":"The paper reports 64 from-scratch pretraining runs of Transformers and three linear-model families at 1.3–8B parameters, trained on 1–2T tokens.","excerpt":"We study it with 64 from-scratch pretraining runs of Transformers and three linear-model families at 1.3-8B parameters on 1-2T Nemotron-CC tokens","support":"source-stated"},{"text":"At 8B, Canon layers improved Transformers, requiring about 30% fewer pretraining tokens for the same MMLU accuracy.","excerpt":"Canon layers, which add local token mixing, consistently improve Transformers, saving about 30% of pretraining tokens for the same MMLU accuracy at 8B.","support":"source-stated"},{"text":"With Canon, gated linear attention matched Gated DeltaNet and outperformed Mamba2 in the reported 8B/1T comparison.","excerpt":"with Canon, the simplest linear model, gated linear attention, matches Gated DeltaNet and outperforms Mamba2 even at 8B/1T.","support":"source-stated"},{"text":"The paper reports that benchmark performance did not imply reliable retrieval from a short context or success on the tested two-hop question.","excerpt":"linear models that match or beat Transformers on these benchmarks cannot, in fact, reliably retrieve information from a short 500-token context. Moreover, no model, Transformer or linear, reliably beats random guessing on arguably the simplest two-hop question","support":"source-stated"},{"text":"Allen-Zhu recommends including in-context reasoning examples in pretraining data early, rather than reserving them for later training stages.","excerpt":"include in-context reasoning examples in pretraining data from an early stage, rather than reserve them for later mid-training, let alone post-training.","support":"source-stated"}],"limitations":["The abstract describes experiments at 1.3–8B parameters and 1–2T tokens; it does not establish that the reported results generalize beyond those tested scales and training setup.","The retrieval and two-hop findings are described as reliability results in the paper’s tested tasks; they do not establish performance on all contexts or reasoning tasks.","The supplied evidence is the arXiv abstract, not the full paper, so it does not provide the detailed experimental methods or task definitions."],"mediaId":"940e7b406941a5264c0d713386adc3e44b3f31325bb7109962bcc2fe8160d684","correctionReason":null,"id":"0fe488ea7cb42fdb7445e8431697520d90aef8b7260f52d9a9c61ef4338413ab","publicationId":"2e8ab8f9-45fd-415d-8af9-367cc89625f0","version":1,"sourceUrl":"https://arxiv.org/abs/2611.24963","sourceTitle":"Physics of Language Models: Part 4.2, Canon Layers at Scale where Synthetic Pretraining Resonates in Reality","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-12","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-20T05:01:18.681Z","createdAt":"2026-11-20T09:48:23.246Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"940e7b406941a5264c0d713386adc3e44b3f31325bb7109962bcc2fe8160d684","kind":"article","url":"https://arxiv.org/abs/2611.24963","sourceUrl":"https://arxiv.org/abs/2611.24963","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-20T09:47:26.529Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.24963","checkedAt":"2026-11-21T02:52:10.059Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"940e7b406941a5264c0d713386adc3e44b3f31325bb7109962bcc2fe8160d684","confidence":0.61,"provider":"typesafe","model":"jev-1.13.0","inputHash":"12c50735f36a79d666cf38bdb7ad21cffc9606ea23e66ab5785ab417e8505823","decidedAt":"2026-11-20T09:47:27.254Z"},"provenance":{"textHash":"1ead7cac1f9cc92ef216ec97a050f6cdc1f4a59615d73985795afaedf364b3d3","observationId":"11971c8922f7f3f91186b4fcab5fefcd123a59f6f16ad96d02ad79c13c152c7c","retrievedAt":"2026-11-20T09:47:26.439Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"234ee2ef-015a-4b7b-8dfd-4652f1568c26"},"reviewer":{"model":"gpt-6-luna","runId":"f82413cc-f294-4153-8cf6-4dc8167002a6"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-20T09:48:23.246Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Allen-Zhu, Zeyuan","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Allen-Zhu, Zeyuan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Allen-Zhu, Zeyuan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24963","locator":"head/meta[name=author|citation_author]"},"assertionId":"84a7d30b0779a10db2250cc1f8bab465531bc8712f4f2dc38666477fc264d7e4:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-21T02:52:10.059Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"84a7d30b0779a10db2250cc1f8bab465531bc8712f4f2dc38666477fc264d7e4","sourceUrl":"https://arxiv.org/abs/2611.24963","contentSha256":"9f76ff4e6aa94722f71b402e6af875032a1c6d66a5bac8974ba874a2e517bce2","textHash":"1ead7cac1f9cc92ef216ec97a050f6cdc1f4a59615d73985795afaedf364b3d3","observedAt":"2026-11-21T02:52:10.059Z"}}],"participationCoverage":{"total":1,"returned":1,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.24963","snapshotId":"84a7d30b0779a10db2250cc1f8bab465531bc8712f4f2dc38666477fc264d7e4","contentSha256":"9f76ff4e6aa94722f71b402e6af875032a1c6d66a5bac8974ba874a2e517bce2","textHash":"1ead7cac1f9cc92ef216ec97a050f6cdc1f4a59615d73985795afaedf364b3d3","observedAt":"2026-11-21T02:52:10.059Z","basis":"publisher-page-metadata","sourceLinks":40,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/0fe488ea7cb42fdb7445e8431697520d90aef8b7260f52d9a9c61ef4338413ab/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-zeyuan-allen-zhu-says-standard-benchmarks-miss-0fe488ea","formats":{"html":"/readings/arxiv-cs-ai-zeyuan-allen-zhu-says-standard-benchmarks-miss-0fe488ea","markdown":"/readings/arxiv-cs-ai-zeyuan-allen-zhu-says-standard-benchmarks-miss-0fe488ea.md","json":"/readings/arxiv-cs-ai-zeyuan-allen-zhu-says-standard-benchmarks-miss-0fe488ea.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"},{"id":"term-pre-training","name":"Pre-training","explanation":"A model's initial training on a large dataset, before further adaptation for particular uses.","referenceUrl":"https://developers.google.com/machine-learning/glossary#pre-training"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"0fe488ea7cb42fdb7445e8431697520d90aef8b7260f52d9a9c61ef4338413ab","revision":"2e8ab8f9-45fd-415d-8af9-367cc89625f0","revisionKind":"publication-id","title":"arXiv cs.AI: Zeyuan Allen-Zhu says standard benchmarks miss model differences and weaknesses","url":"https://escapemeridian.com/readings/arxiv-cs-ai-zeyuan-allen-zhu-says-standard-benchmarks-miss-0fe488ea"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.24963","revision":"1ead7cac1f9cc92ef216ec97a050f6cdc1f4a59615d73985795afaedf364b3d3","revisionKind":"sha256-source-text","title":"Physics of Language Models: Part 4.2, Canon Layers at Scale where Synthetic Pretraining Resonates in Reality","url":"https://arxiv.org/abs/2611.24963"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter."},"revisions":[{"id":"0fe488ea7cb42fdb7445e8431697520d90aef8b7260f52d9a9c61ef4338413ab","title":"arXiv cs.AI: Zeyuan Allen-Zhu says standard benchmarks miss model differences and weaknesses","argument":"In the arXiv abstract, Zeyuan Allen-Zhu reports that architectural differences can be hard to detect at academic-scale pretraining, but become measurable at larger scales. Canon layers add local token mixing and, the paper reports, improve Transformers while using about 30% fewer pretraining tokens for the same MMLU accuracy at 8B. With Canon, gated linear attention matches Gated DeltaNet and outperforms Mamba2 in the reported comparison. Allen-Zhu suggests that some architectural machinery may mainly mix neighboring tokens.\n\nThe paper also argues that standard benchmark scores can hide weaknesses in basic in-context skills. Linear models that match or beat Transformers on these benchmarks cannot reliably retrieve information from a short 500-token context. No tested model reliably beats random guessing on the paper’s two-hop question. Allen-Zhu says small synthetic playgrounds predicted architectural differences that standard benchmarks missed. He recommends including in-context reasoning examples in pretraining data early, rather than reserving them for later training stages. The findings challenge research based primarily on standard benchmark scores; the abstract reports experiments at particular scales and does not establish how broadly the results generalize.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract describes experiments at 1.3–8B parameters and 1–2T tokens; it does not establish that the reported results generalize beyond those tested scales and training setup. The retrieval and two-hop findings are described as reliability results in the paper’s tested tasks; they do not establish performance on all contexts or reasoning tasks. The supplied evidence is the arXiv abstract, not the full paper, so it does not provide the detailed experimental methods or task definitions.","speaker":{"name":"Allen-Zhu, Zeyuan","roleAtSource":"Article author"},"source":{"title":"Physics of Language Models: Part 4.2, Canon Layers at Scale where Synthetic Pretraining Resonates in Reality","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.24963","publishedAt":"2026-11-12","dateBasis":"Date basis not provided."},"evidence":[{"text":"The paper reports 64 from-scratch pretraining runs of Transformers and three linear-model families at 1.3–8B parameters, trained on 1–2T tokens.","excerpt":"We study it with 64 from-scratch pretraining runs of Transformers and three linear-model families at 1.3-8B parameters on 1-2T Nemotron-CC tokens","url":"https://arxiv.org/abs/2611.24963","relationship":"Source-stated"},{"text":"At 8B, Canon layers improved Transformers, requiring about 30% fewer pretraining tokens for the same MMLU accuracy.","excerpt":"Canon layers, which add local token mixing, consistently improve Transformers, saving about 30% of pretraining tokens for the same MMLU accuracy at 8B.","url":"https://arxiv.org/abs/2611.24963","relationship":"Source-stated"},{"text":"With Canon, gated linear attention matched Gated DeltaNet and outperformed Mamba2 in the reported 8B/1T comparison.","excerpt":"with Canon, the simplest linear model, gated linear attention, matches Gated DeltaNet and outperforms Mamba2 even at 8B/1T.","url":"https://arxiv.org/abs/2611.24963","relationship":"Source-stated"},{"text":"The paper reports that benchmark performance did not imply reliable retrieval from a short context or success on the tested two-hop question.","excerpt":"linear models that match or beat Transformers on these benchmarks cannot, in fact, reliably retrieve information from a short 500-token context. Moreover, no model, Transformer or linear, reliably beats random guessing on arguably the simplest two-hop question","url":"https://arxiv.org/abs/2611.24963","relationship":"Source-stated"},{"text":"Allen-Zhu recommends including in-context reasoning examples in pretraining data early, rather than reserving them for later training stages.","excerpt":"include in-context reasoning examples in pretraining data from an early stage, rather than reserve them for later mid-training, let alone post-training.","url":"https://arxiv.org/abs/2611.24963","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.24963","checkedAt":"2026-11-21T02:52:10.059Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Would including in-context reasoning examples in pretraining data early improve performance on the short-context retrieval and two-hop tasks described in the paper?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Zeyuan Allen-Zhu says standard benchmarks miss model differences and weaknesses","summary":"In the arXiv abstract, Zeyuan Allen-Zhu reports that architectural differences can be hard to detect at academic-scale pretraining, but become measurable at larger scales. Canon layers add local token mixing and, the paper reports, improve Transformers while using about 30% fewer pretraining tokens for the same MMLU accuracy at 8B. With Canon, gated linear attention matches Gated DeltaNet and outperforms Mamba2 in the reported comparison. Allen-Zhu suggests that some architectural machinery may mainly mix neighboring tokens.\n\nThe paper also argues that standard benchmark scores can hide weaknesses in basic in-context skills. Linear models that match or beat Transformers on these benchmarks cannot reliably retrieve information from a short 500-token context. No tested model reliably beats random guessing on the paper’s two-hop question. Allen-Zhu says small synthetic playgrounds predicted architectural differences that standard benchmarks missed. He recommends including in-context reasoning examples in pretraining data early, rather than reserving them for later training stages. The findings challenge research based primarily on standard benchmark scores; the abstract reports experiments at particular scales and does not establish how broadly the results generalize.","question":"Would including in-context reasoning examples in pretraining data early improve performance on the short-context retrieval and two-hop tasks described in the paper?","claims":[{"text":"The paper reports 64 from-scratch pretraining runs of Transformers and three linear-model families at 1.3–8B parameters, trained on 1–2T tokens.","excerpt":"We study it with 64 from-scratch pretraining runs of Transformers and three linear-model families at 1.3-8B parameters on 1-2T Nemotron-CC tokens","support":"source-stated"},{"text":"At 8B, Canon layers improved Transformers, requiring about 30% fewer pretraining tokens for the same MMLU accuracy.","excerpt":"Canon layers, which add local token mixing, consistently improve Transformers, saving about 30% of pretraining tokens for the same MMLU accuracy at 8B.","support":"source-stated"},{"text":"With Canon, gated linear attention matched Gated DeltaNet and outperformed Mamba2 in the reported 8B/1T comparison.","excerpt":"with Canon, the simplest linear model, gated linear attention, matches Gated DeltaNet and outperforms Mamba2 even at 8B/1T.","support":"source-stated"},{"text":"The paper reports that benchmark performance did not imply reliable retrieval from a short context or success on the tested two-hop question.","excerpt":"linear models that match or beat Transformers on these benchmarks cannot, in fact, reliably retrieve information from a short 500-token context. Moreover, no model, Transformer or linear, reliably beats random guessing on arguably the simplest two-hop question","support":"source-stated"},{"text":"Allen-Zhu recommends including in-context reasoning examples in pretraining data early, rather than reserving them for later training stages.","excerpt":"include in-context reasoning examples in pretraining data from an early stage, rather than reserve them for later mid-training, let alone post-training.","support":"source-stated"}],"limitations":["The abstract describes experiments at 1.3–8B parameters and 1–2T tokens; it does not establish that the reported results generalize beyond those tested scales and training setup.","The retrieval and two-hop findings are described as reliability results in the paper’s tested tasks; they do not establish performance on all contexts or reasoning tasks.","The supplied evidence is the arXiv abstract, not the full paper, so it does not provide the detailed experimental methods or task definitions."],"mediaId":"940e7b406941a5264c0d713386adc3e44b3f31325bb7109962bcc2fe8160d684","correctionReason":null,"id":"0fe488ea7cb42fdb7445e8431697520d90aef8b7260f52d9a9c61ef4338413ab","publicationId":"2e8ab8f9-45fd-415d-8af9-367cc89625f0","version":1,"sourceUrl":"https://arxiv.org/abs/2611.24963","sourceTitle":"Physics of Language Models: Part 4.2, Canon Layers at Scale where Synthetic Pretraining Resonates in Reality","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-12","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-20T05:01:18.681Z","createdAt":"2026-11-20T09:48:23.246Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"940e7b406941a5264c0d713386adc3e44b3f31325bb7109962bcc2fe8160d684","kind":"article","url":"https://arxiv.org/abs/2611.24963","sourceUrl":"https://arxiv.org/abs/2611.24963","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-20T09:47:26.529Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.24963","checkedAt":"2026-11-21T02:52:10.059Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"940e7b406941a5264c0d713386adc3e44b3f31325bb7109962bcc2fe8160d684","confidence":0.61,"provider":"typesafe","model":"jev-1.13.0","inputHash":"12c50735f36a79d666cf38bdb7ad21cffc9606ea23e66ab5785ab417e8505823","decidedAt":"2026-11-20T09:47:27.254Z"},"provenance":{"textHash":"1ead7cac1f9cc92ef216ec97a050f6cdc1f4a59615d73985795afaedf364b3d3","observationId":"11971c8922f7f3f91186b4fcab5fefcd123a59f6f16ad96d02ad79c13c152c7c","retrievedAt":"2026-11-20T09:47:26.439Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"234ee2ef-015a-4b7b-8dfd-4652f1568c26"},"reviewer":{"model":"gpt-6-luna","runId":"f82413cc-f294-4153-8cf6-4dc8167002a6"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-20T09:48:23.246Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Allen-Zhu, Zeyuan","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Allen-Zhu, Zeyuan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Allen-Zhu, Zeyuan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24963","locator":"head/meta[name=author|citation_author]"},"assertionId":"84a7d30b0779a10db2250cc1f8bab465531bc8712f4f2dc38666477fc264d7e4:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-21T02:52:10.059Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"84a7d30b0779a10db2250cc1f8bab465531bc8712f4f2dc38666477fc264d7e4","sourceUrl":"https://arxiv.org/abs/2611.24963","contentSha256":"9f76ff4e6aa94722f71b402e6af875032a1c6d66a5bac8974ba874a2e517bce2","textHash":"1ead7cac1f9cc92ef216ec97a050f6cdc1f4a59615d73985795afaedf364b3d3","observedAt":"2026-11-21T02:52:10.059Z"}}],"participationCoverage":{"total":1,"returned":1,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.24963","snapshotId":"84a7d30b0779a10db2250cc1f8bab465531bc8712f4f2dc38666477fc264d7e4","contentSha256":"9f76ff4e6aa94722f71b402e6af875032a1c6d66a5bac8974ba874a2e517bce2","textHash":"1ead7cac1f9cc92ef216ec97a050f6cdc1f4a59615d73985795afaedf364b3d3","observedAt":"2026-11-21T02:52:10.059Z","basis":"publisher-page-metadata","sourceLinks":40,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/0fe488ea7cb42fdb7445e8431697520d90aef8b7260f52d9a9c61ef4338413ab/source"}}}],"audits":[{"publicationId":"2e8ab8f9-45fd-415d-8af9-367cc89625f0","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"Zeyuan Allen-Zhu is correctly named as sole author, and the arXiv venue is correct. Most findings stay attributed. One small blur: the closing sentence states 'The findings challenge research based primarily on standard benchmark scores' in the reading's own voice. The source asserts this in bold. Smallest repair: 'Allen-Zhu argues the findings challenge…'.","excerpt":"These findings $\\textbf{challenge traditional architecture research}$ based primarily on standard benchmark scores."},{"dimension":"summary","status":"uncertain","explanation":"The core argument is preserved, but central terms go unexplained for a non-specialist: Transformers vs linear models, gated linear attention, Gated DeltaNet, Mamba2, MMLU, local token mixing, two-hop question, in-context retrieval and pretraining tokens. These should be kept exactly but explained, for example via tooltips. Two points lose force: the source's pointed claim that Gated DeltaNet's and Mamba2's extra machinery 'may do little more than mix neighboring tokens' is flattened to 'some architectural machinery'. The detail that synthetic playgrounds predicted the differences at under 100M parameters is dropped. Omitting GLA5/GDN2 and the roughly 100k USD cost is minor.","excerpt":"Therefore, the extra machinery of Gated DeltaNet and Mamba2 may do little more than mix neighboring tokens."},{"dimension":"date","status":"pass","explanation":"originalPublished 2026-11-12 matches the arXiv v1 submission date. The comments note that results, recipes and code were public at an external URL since December 9, 2025. That is a separate earlier release, not this paper's first submission. Readers may still benefit from that context.","excerpt":"[Submitted on 12 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"All five claims are source-stated, and their excerpts are verbatim. The 30% token saving is correctly scoped to MMLU at 8B. The GLA vs Gated DeltaNet/Mamba2 result is scoped to 8B/1T with Canon. The retrieval and two-hop results are scoped to the tested tasks. The recommendation is attributed as the author's. The question raises an open link the abstract does not establish, rather than asserting it. The limitations appropriately note that only the abstract is supplied.","excerpt":"For practitioners, one lesson is to include in-context reasoning examples in pretraining data from an early stage, rather than reserve them for later mid-training, let alone post-training."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper. It matches the source URL and is the most direct access to the original.","excerpt":null}],"createdAt":"2026-11-20T09:48:23.246Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}