{"id":"55b661b7ad319aabcb33d9312753b9fb9816748f4eefb96ffabfad7d689187c0","title":"arXiv cs.AI paper introduces Dreamer, a depth-recurrent attention mixture for latent reasoning","argument":"The authors of Dreamer argue that depth-recurrent models can use more steps for latent reasoning if attention is applied along depth, rather than relying on a fixed hidden size. Their framework combines sequence attention, depth attention and sparse expert attention; the abstract says this addresses limits in prior comparisons and model design. In plain terms, the authors’ proposal lets a model allocate attention across its successive processing depths, alongside attention across the input sequence and among experts. They report that, on language reasoning benchmarks, Dreamer models need 2 to 8x fewer training tokens to reach the same accuracy as FLOP-, parameter- and memory-matched state-of-the-art models, and outperform roughly 2x larger state-of-the-art models given the same training tokens. They also report 2 to 11x larger expert selection diversity than state-of-the-art mixture-of-experts models. These are results reported by the paper’s authors, not evidence here of performance beyond the stated benchmarks. The abstract does not provide benchmark names or experimental details, so the supplied evidence cannot show how broadly the results apply. The paper also says its analysis offers insights into how models use knowledge across depths, but the abstract does not describe those findings in detail.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied abstract does not name the language reasoning benchmarks or provide experimental details, so the reported results cannot be assessed or generalized from this evidence alone. The abstract says the paper presents insights into knowledge usage across depths but does not describe those findings in detail.","speaker":{"name":"Knupp, Jonas, Metzen, Jan Hendrik, Bohn, Jeremias, Groh, Georg, Kersting, Kristian","roleAtSource":"Article author"},"source":{"title":"Depth-Recurrent Attention Mixture (Dreamer): Giving Latent Reasoning the Attention it Deserves","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2601.21582","publishedAt":"2026-01-29","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors introduce Dreamer, a framework combining sequence attention, depth attention and sparse expert attention to address limitations they identify in prior depth-recurrent models.","excerpt":"we introduce a modular framework of depth-recurrent attention mixtures (Dreamer), combining sequence attention, depth attention, and sparse expert attention.","url":"https://arxiv.org/abs/2601.21582","relationship":"Source-stated"},{"text":"On language reasoning benchmarks, the authors report that their models need 2 to 8x fewer training tokens to reach the same accuracy as matched state-of-the-art models.","excerpt":"Across language reasoning benchmarks, our models require 2 to 8x fewer training tokens for the same accuracy as FLOP-, parameter-, and memory-matched SOTA","url":"https://arxiv.org/abs/2601.21582","relationship":"Source-stated"},{"text":"The authors report that Dreamer outperforms state-of-the-art models about twice its size when given the same number of training tokens.","excerpt":"and outperform ca. 2x larger SOTA models with the same training tokens.","url":"https://arxiv.org/abs/2601.21582","relationship":"Source-stated"},{"text":"The authors report greater expert selection diversity than in state-of-the-art mixture-of-experts models.","excerpt":"showing 2 to 11x larger expert selection diversity than SOTA MoEs.","url":"https://arxiv.org/abs/2601.21582","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2601.21582","checkedAt":"2026-11-09T07:40:07.189Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Which language reasoning benchmarks and experimental details support the reported results, and do they hold beyond those benchmarks?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI paper introduces Dreamer, a depth-recurrent attention mixture for latent reasoning","summary":"The authors of Dreamer argue that depth-recurrent models can use more steps for latent reasoning if attention is applied along depth, rather than relying on a fixed hidden size. Their framework combines sequence attention, depth attention and sparse expert attention; the abstract says this addresses limits in prior comparisons and model design. In plain terms, the authors’ proposal lets a model allocate attention across its successive processing depths, alongside attention across the input sequence and among experts. They report that, on language reasoning benchmarks, Dreamer models need 2 to 8x fewer training tokens to reach the same accuracy as FLOP-, parameter- and memory-matched state-of-the-art models, and outperform roughly 2x larger state-of-the-art models given the same training tokens. They also report 2 to 11x larger expert selection diversity than state-of-the-art mixture-of-experts models. These are results reported by the paper’s authors, not evidence here of performance beyond the stated benchmarks. The abstract does not provide benchmark names or experimental details, so the supplied evidence cannot show how broadly the results apply. The paper also says its analysis offers insights into how models use knowledge across depths, but the abstract does not describe those findings in detail.","question":"Which language reasoning benchmarks and experimental details support the reported results, and do they hold beyond those benchmarks?","claims":[{"text":"The authors introduce Dreamer, a framework combining sequence attention, depth attention and sparse expert attention to address limitations they identify in prior depth-recurrent models.","excerpt":"we introduce a modular framework of depth-recurrent attention mixtures (Dreamer), combining sequence attention, depth attention, and sparse expert attention.","support":"source-stated"},{"text":"On language reasoning benchmarks, the authors report that their models need 2 to 8x fewer training tokens to reach the same accuracy as matched state-of-the-art models.","excerpt":"Across language reasoning benchmarks, our models require 2 to 8x fewer training tokens for the same accuracy as FLOP-, parameter-, and memory-matched SOTA","support":"source-stated"},{"text":"The authors report that Dreamer outperforms state-of-the-art models about twice its size when given the same number of training tokens.","excerpt":"and outperform ca. 2x larger SOTA models with the same training tokens.","support":"source-stated"},{"text":"The authors report greater expert selection diversity than in state-of-the-art mixture-of-experts models.","excerpt":"showing 2 to 11x larger expert selection diversity than SOTA MoEs.","support":"source-stated"}],"limitations":["The supplied abstract does not name the language reasoning benchmarks or provide experimental details, so the reported results cannot be assessed or generalized from this evidence alone.","The abstract says the paper presents insights into knowledge usage across depths but does not describe those findings in detail."],"mediaId":"dd4fcb0c5c010dcc16c22c99a1faf306832ac6f1d1ebc660cfdec86eef93db92","correctionReason":null,"id":"55b661b7ad319aabcb33d9312753b9fb9816748f4eefb96ffabfad7d689187c0","publicationId":"e20e0cd4-338c-4f31-8694-913001463199","version":1,"sourceUrl":"https://arxiv.org/abs/2601.21582","sourceTitle":"Depth-Recurrent Attention Mixture (Dreamer): Giving Latent Reasoning the Attention it Deserves","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-01-29","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-06T05:07:31.376Z","createdAt":"2026-11-08T23:08:35.475Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"dd4fcb0c5c010dcc16c22c99a1faf306832ac6f1d1ebc660cfdec86eef93db92","kind":"article","url":"https://arxiv.org/abs/2601.21582","sourceUrl":"https://arxiv.org/abs/2601.21582","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-08T23:03:47.236Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2601.21582","checkedAt":"2026-11-09T07:40:07.189Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"dd4fcb0c5c010dcc16c22c99a1faf306832ac6f1d1ebc660cfdec86eef93db92","confidence":0.89,"provider":"typesafe","model":"jev-1.13.0","inputHash":"3ffda4a08b2d9d699a77ff5da5e9ce45598d6be342b171f4670f69b556a90a56","decidedAt":"2026-11-08T23:03:48.135Z"},"provenance":{"textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observationId":"83f00a627afc7e55a56f32b639da2145c51ec71625d6446d42d51d5c3674b337","retrievedAt":"2026-11-08T23:03:47.082Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"44fbf60c-63ec-4001-8815-2d31f3c58709"},"reviewer":{"model":"gpt-6-luna","runId":"c1b9533a-d02e-48f8-864e-409e6df9b67b"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-08T23:08:35.475Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Knupp, Jonas","url":null,"kind":"unknown"},{"name":"Metzen, Jan Hendrik","url":null,"kind":"unknown"},{"name":"Bohn, Jeremias","url":null,"kind":"unknown"},{"name":"Groh, Georg","url":null,"kind":"unknown"},{"name":"Kersting, Kristian","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Knupp, Jonas","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Knupp, Jonas","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2601.21582","locator":"head/meta[name=author|citation_author]"},"assertionId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:40:07.189Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8","sourceUrl":"https://arxiv.org/abs/2601.21582","contentSha256":"575417a0dedf74d4efe3b630a721efd063dad7b6dacc76282991e62a007f158f","textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observedAt":"2026-11-09T07:40:07.189Z"}},{"actor":{"name":"Metzen, Jan Hendrik","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Metzen, Jan Hendrik","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2601.21582","locator":"head/meta[name=author|citation_author]"},"assertionId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:40:07.189Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8","sourceUrl":"https://arxiv.org/abs/2601.21582","contentSha256":"575417a0dedf74d4efe3b630a721efd063dad7b6dacc76282991e62a007f158f","textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observedAt":"2026-11-09T07:40:07.189Z"}},{"actor":{"name":"Bohn, Jeremias","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Bohn, Jeremias","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2601.21582","locator":"head/meta[name=author|citation_author]"},"assertionId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:40:07.189Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8","sourceUrl":"https://arxiv.org/abs/2601.21582","contentSha256":"575417a0dedf74d4efe3b630a721efd063dad7b6dacc76282991e62a007f158f","textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observedAt":"2026-11-09T07:40:07.189Z"}},{"actor":{"name":"Groh, Georg","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Groh, Georg","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2601.21582","locator":"head/meta[name=author|citation_author]"},"assertionId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:40:07.189Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8","sourceUrl":"https://arxiv.org/abs/2601.21582","contentSha256":"575417a0dedf74d4efe3b630a721efd063dad7b6dacc76282991e62a007f158f","textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observedAt":"2026-11-09T07:40:07.189Z"}},{"actor":{"name":"Kersting, Kristian","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Kersting, Kristian","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2601.21582","locator":"head/meta[name=author|citation_author]"},"assertionId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:40:07.189Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8","sourceUrl":"https://arxiv.org/abs/2601.21582","contentSha256":"575417a0dedf74d4efe3b630a721efd063dad7b6dacc76282991e62a007f158f","textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observedAt":"2026-11-09T07:40:07.189Z"}}],"participationCoverage":{"total":5,"returned":5,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2601.21582","snapshotId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8","contentSha256":"575417a0dedf74d4efe3b630a721efd063dad7b6dacc76282991e62a007f158f","textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observedAt":"2026-11-09T07:40:07.189Z","basis":"publisher-page-metadata","sourceLinks":47,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/55b661b7ad319aabcb33d9312753b9fb9816748f4eefb96ffabfad7d689187c0/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-paper-introduces-dreamer-a-depth-recurrent-55b661b7","formats":{"html":"/readings/arxiv-cs-ai-paper-introduces-dreamer-a-depth-recurrent-55b661b7","markdown":"/readings/arxiv-cs-ai-paper-introduces-dreamer-a-depth-recurrent-55b661b7.md","json":"/readings/arxiv-cs-ai-paper-introduces-dreamer-a-depth-recurrent-55b661b7.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"},{"id":"term-flops","name":"Floating-point operations","explanation":"A count of basic arithmetic steps; used to measure how much computation went into training a model.","referenceUrl":"https://en.wikipedia.org/wiki/FLOPS"},{"id":"term-mixture-of-experts","name":"Mixture of experts","explanation":"A model design that sends each input to a few specialized sub-networks instead of the whole model, which saves computation.","referenceUrl":"https://developers.google.com/machine-learning/glossary#mixture-of-experts"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"55b661b7ad319aabcb33d9312753b9fb9816748f4eefb96ffabfad7d689187c0","revision":"e20e0cd4-338c-4f31-8694-913001463199","revisionKind":"publication-id","title":"arXiv cs.AI paper introduces Dreamer, a depth-recurrent attention mixture for latent reasoning","url":"https://escapemeridian.com/readings/arxiv-cs-ai-paper-introduces-dreamer-a-depth-recurrent-55b661b7"},"source":{"type":"source","id":"https://arxiv.org/abs/2601.21582","revision":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","revisionKind":"sha256-source-text","title":"Depth-Recurrent Attention Mixture (Dreamer): Giving Latent Reasoning the Attention it Deserves","url":"https://arxiv.org/abs/2601.21582"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"55b661b7ad319aabcb33d9312753b9fb9816748f4eefb96ffabfad7d689187c0","title":"arXiv cs.AI paper introduces Dreamer, a depth-recurrent attention mixture for latent reasoning","argument":"The authors of Dreamer argue that depth-recurrent models can use more steps for latent reasoning if attention is applied along depth, rather than relying on a fixed hidden size. Their framework combines sequence attention, depth attention and sparse expert attention; the abstract says this addresses limits in prior comparisons and model design. In plain terms, the authors’ proposal lets a model allocate attention across its successive processing depths, alongside attention across the input sequence and among experts. They report that, on language reasoning benchmarks, Dreamer models need 2 to 8x fewer training tokens to reach the same accuracy as FLOP-, parameter- and memory-matched state-of-the-art models, and outperform roughly 2x larger state-of-the-art models given the same training tokens. They also report 2 to 11x larger expert selection diversity than state-of-the-art mixture-of-experts models. These are results reported by the paper’s authors, not evidence here of performance beyond the stated benchmarks. The abstract does not provide benchmark names or experimental details, so the supplied evidence cannot show how broadly the results apply. The paper also says its analysis offers insights into how models use knowledge across depths, but the abstract does not describe those findings in detail.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied abstract does not name the language reasoning benchmarks or provide experimental details, so the reported results cannot be assessed or generalized from this evidence alone. The abstract says the paper presents insights into knowledge usage across depths but does not describe those findings in detail.","speaker":{"name":"Knupp, Jonas, Metzen, Jan Hendrik, Bohn, Jeremias, Groh, Georg, Kersting, Kristian","roleAtSource":"Article author"},"source":{"title":"Depth-Recurrent Attention Mixture (Dreamer): Giving Latent Reasoning the Attention it Deserves","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2601.21582","publishedAt":"2026-01-29","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors introduce Dreamer, a framework combining sequence attention, depth attention and sparse expert attention to address limitations they identify in prior depth-recurrent models.","excerpt":"we introduce a modular framework of depth-recurrent attention mixtures (Dreamer), combining sequence attention, depth attention, and sparse expert attention.","url":"https://arxiv.org/abs/2601.21582","relationship":"Source-stated"},{"text":"On language reasoning benchmarks, the authors report that their models need 2 to 8x fewer training tokens to reach the same accuracy as matched state-of-the-art models.","excerpt":"Across language reasoning benchmarks, our models require 2 to 8x fewer training tokens for the same accuracy as FLOP-, parameter-, and memory-matched SOTA","url":"https://arxiv.org/abs/2601.21582","relationship":"Source-stated"},{"text":"The authors report that Dreamer outperforms state-of-the-art models about twice its size when given the same number of training tokens.","excerpt":"and outperform ca. 2x larger SOTA models with the same training tokens.","url":"https://arxiv.org/abs/2601.21582","relationship":"Source-stated"},{"text":"The authors report greater expert selection diversity than in state-of-the-art mixture-of-experts models.","excerpt":"showing 2 to 11x larger expert selection diversity than SOTA MoEs.","url":"https://arxiv.org/abs/2601.21582","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2601.21582","checkedAt":"2026-11-09T07:40:07.189Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Which language reasoning benchmarks and experimental details support the reported results, and do they hold beyond those benchmarks?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI paper introduces Dreamer, a depth-recurrent attention mixture for latent reasoning","summary":"The authors of Dreamer argue that depth-recurrent models can use more steps for latent reasoning if attention is applied along depth, rather than relying on a fixed hidden size. Their framework combines sequence attention, depth attention and sparse expert attention; the abstract says this addresses limits in prior comparisons and model design. In plain terms, the authors’ proposal lets a model allocate attention across its successive processing depths, alongside attention across the input sequence and among experts. They report that, on language reasoning benchmarks, Dreamer models need 2 to 8x fewer training tokens to reach the same accuracy as FLOP-, parameter- and memory-matched state-of-the-art models, and outperform roughly 2x larger state-of-the-art models given the same training tokens. They also report 2 to 11x larger expert selection diversity than state-of-the-art mixture-of-experts models. These are results reported by the paper’s authors, not evidence here of performance beyond the stated benchmarks. The abstract does not provide benchmark names or experimental details, so the supplied evidence cannot show how broadly the results apply. The paper also says its analysis offers insights into how models use knowledge across depths, but the abstract does not describe those findings in detail.","question":"Which language reasoning benchmarks and experimental details support the reported results, and do they hold beyond those benchmarks?","claims":[{"text":"The authors introduce Dreamer, a framework combining sequence attention, depth attention and sparse expert attention to address limitations they identify in prior depth-recurrent models.","excerpt":"we introduce a modular framework of depth-recurrent attention mixtures (Dreamer), combining sequence attention, depth attention, and sparse expert attention.","support":"source-stated"},{"text":"On language reasoning benchmarks, the authors report that their models need 2 to 8x fewer training tokens to reach the same accuracy as matched state-of-the-art models.","excerpt":"Across language reasoning benchmarks, our models require 2 to 8x fewer training tokens for the same accuracy as FLOP-, parameter-, and memory-matched SOTA","support":"source-stated"},{"text":"The authors report that Dreamer outperforms state-of-the-art models about twice its size when given the same number of training tokens.","excerpt":"and outperform ca. 2x larger SOTA models with the same training tokens.","support":"source-stated"},{"text":"The authors report greater expert selection diversity than in state-of-the-art mixture-of-experts models.","excerpt":"showing 2 to 11x larger expert selection diversity than SOTA MoEs.","support":"source-stated"}],"limitations":["The supplied abstract does not name the language reasoning benchmarks or provide experimental details, so the reported results cannot be assessed or generalized from this evidence alone.","The abstract says the paper presents insights into knowledge usage across depths but does not describe those findings in detail."],"mediaId":"dd4fcb0c5c010dcc16c22c99a1faf306832ac6f1d1ebc660cfdec86eef93db92","correctionReason":null,"id":"55b661b7ad319aabcb33d9312753b9fb9816748f4eefb96ffabfad7d689187c0","publicationId":"e20e0cd4-338c-4f31-8694-913001463199","version":1,"sourceUrl":"https://arxiv.org/abs/2601.21582","sourceTitle":"Depth-Recurrent Attention Mixture (Dreamer): Giving Latent Reasoning the Attention it Deserves","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-01-29","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-06T05:07:31.376Z","createdAt":"2026-11-08T23:08:35.475Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"dd4fcb0c5c010dcc16c22c99a1faf306832ac6f1d1ebc660cfdec86eef93db92","kind":"article","url":"https://arxiv.org/abs/2601.21582","sourceUrl":"https://arxiv.org/abs/2601.21582","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-08T23:03:47.236Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2601.21582","checkedAt":"2026-11-09T07:40:07.189Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"dd4fcb0c5c010dcc16c22c99a1faf306832ac6f1d1ebc660cfdec86eef93db92","confidence":0.89,"provider":"typesafe","model":"jev-1.13.0","inputHash":"3ffda4a08b2d9d699a77ff5da5e9ce45598d6be342b171f4670f69b556a90a56","decidedAt":"2026-11-08T23:03:48.135Z"},"provenance":{"textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observationId":"83f00a627afc7e55a56f32b639da2145c51ec71625d6446d42d51d5c3674b337","retrievedAt":"2026-11-08T23:03:47.082Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"44fbf60c-63ec-4001-8815-2d31f3c58709"},"reviewer":{"model":"gpt-6-luna","runId":"c1b9533a-d02e-48f8-864e-409e6df9b67b"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-08T23:08:35.475Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Knupp, Jonas","url":null,"kind":"unknown"},{"name":"Metzen, Jan Hendrik","url":null,"kind":"unknown"},{"name":"Bohn, Jeremias","url":null,"kind":"unknown"},{"name":"Groh, Georg","url":null,"kind":"unknown"},{"name":"Kersting, Kristian","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Knupp, Jonas","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Knupp, Jonas","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2601.21582","locator":"head/meta[name=author|citation_author]"},"assertionId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:40:07.189Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8","sourceUrl":"https://arxiv.org/abs/2601.21582","contentSha256":"575417a0dedf74d4efe3b630a721efd063dad7b6dacc76282991e62a007f158f","textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observedAt":"2026-11-09T07:40:07.189Z"}},{"actor":{"name":"Metzen, Jan Hendrik","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Metzen, Jan Hendrik","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2601.21582","locator":"head/meta[name=author|citation_author]"},"assertionId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:40:07.189Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8","sourceUrl":"https://arxiv.org/abs/2601.21582","contentSha256":"575417a0dedf74d4efe3b630a721efd063dad7b6dacc76282991e62a007f158f","textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observedAt":"2026-11-09T07:40:07.189Z"}},{"actor":{"name":"Bohn, Jeremias","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Bohn, Jeremias","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2601.21582","locator":"head/meta[name=author|citation_author]"},"assertionId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:40:07.189Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8","sourceUrl":"https://arxiv.org/abs/2601.21582","contentSha256":"575417a0dedf74d4efe3b630a721efd063dad7b6dacc76282991e62a007f158f","textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observedAt":"2026-11-09T07:40:07.189Z"}},{"actor":{"name":"Groh, Georg","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Groh, Georg","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2601.21582","locator":"head/meta[name=author|citation_author]"},"assertionId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:40:07.189Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8","sourceUrl":"https://arxiv.org/abs/2601.21582","contentSha256":"575417a0dedf74d4efe3b630a721efd063dad7b6dacc76282991e62a007f158f","textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observedAt":"2026-11-09T07:40:07.189Z"}},{"actor":{"name":"Kersting, Kristian","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Kersting, Kristian","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2601.21582","locator":"head/meta[name=author|citation_author]"},"assertionId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:40:07.189Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8","sourceUrl":"https://arxiv.org/abs/2601.21582","contentSha256":"575417a0dedf74d4efe3b630a721efd063dad7b6dacc76282991e62a007f158f","textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observedAt":"2026-11-09T07:40:07.189Z"}}],"participationCoverage":{"total":5,"returned":5,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2601.21582","snapshotId":"1337a3ddd0672d846ec0b080537c0d2e735125c3f58df4c049c161a84494a3c8","contentSha256":"575417a0dedf74d4efe3b630a721efd063dad7b6dacc76282991e62a007f158f","textHash":"d263c9f388c5de6e0115f341bc751ad8cc65523e10eb4ebdb158189460b5b70d","observedAt":"2026-11-09T07:40:07.189Z","basis":"publisher-page-metadata","sourceLinks":47,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/55b661b7ad319aabcb33d9312753b9fb9816748f4eefb96ffabfad7d689187c0/source"}}}],"audits":[{"publicationId":"e20e0cd4-338c-4f31-8694-913001463199","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The draft attributes the framework and results to the paper's authors on arXiv cs.AI and keeps the performance figures framed as author-reported, not established fact.","excerpt":"Authors:Jonas Knupp, Jan Hendrik Metzen, Jeremias Bohn, Georg Groh, Kristian Kersting"},{"dimension":"summary","status":"uncertain","explanation":"The summary is faithful to the abstract, but a non-specialist would struggle with its central terms. It never defines depth-recurrence, which the abstract explains as sharing parameters across depths, meaning the same layers are reused repeatedly. It also leaves latent reasoning, hidden size, FLOP-matched baselines, training tokens and mixture-of-experts unexplained, with no tooltip markers. The plain-terms sentence helps only partly. Smallest repair: add the source's own definition of depth-recurrence and tooltip the other terms. The question is substantive and grounded.","excerpt":"Depth-recurrence facilitates latent reasoning by sharing parameters across depths."},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-01-29, which matches the v1 submission date. It is not the 5 Nov 2026 revision or the November feed date.","excerpt":"[Submitted on 29 Jan 2026 (v1), last revised 5 Nov 2026 (this version, v2)]"},{"dimension":"support","status":"pass","explanation":"Each claim is quoted from the abstract with the correct numbers (2 to 8x, ca. 2x, 2 to 11x) and their conditions: matched baselines and the same training tokens. The hidden-size framing matches the abstract. The limitations accurately note that benchmark names and experimental details are missing, and no benefits beyond the source are implied.","excerpt":"It alleviates the hidden-size bottleneck through attention along depth, decouples scaling dimensions"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source.","excerpt":null}],"createdAt":"2026-11-08T23:08:35.475Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}