{"id":"84b37ec63bc097ea99cfc92d42494c2135a87ecaad3b29f9cbe50abaf12cc36e","title":"LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning","argument":"Shradha Agarwal, Deepak Rajbhar and Tariq J present LinAlg-Bench, a diagnostic benchmark of 10 frontier large language models solving structured linear algebra problems on 3x3, 4x4 and 5x5 matrices. The authors argue that the models’ errors follow a pattern rather than occurring at random: at 3x3, failures tend to involve execution errors such as sign tracking and arithmetic, while at 4x4 and 5x5, models often abandon computation and produce fabricated responses. They describe this shift as near-universal across model tiers and architectures, and suggest it reflects a working memory limit rather than a knowledge gap. The benchmark uses 660 SymPy-certified problems across 9 task types and analyzes 6,600 outputs; its forensic pipeline classifies failures into error tags and subtypes. The authors also report that solution strategy rigidity is a near-perfect predictor of 5x5 determinant accuracy, and identify constraint-aware confabulation as a structured hallucination failure mode. These findings make the benchmark a way to examine not just whether models get answers wrong, but how their failures change with problem scale. The claims concern this benchmark’s tasks and evaluated models; the supplied abstract does not establish how the findings extend beyond them.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the arXiv abstract, not the paper’s full methods or results; it does not provide the details needed to assess the benchmark construction or the reported analyses. The findings described concern the evaluated models and the benchmark’s structured linear algebra tasks; the abstract does not establish whether the patterns extend to other tasks or settings.","speaker":{"name":"Agarwal, Shradha, Rajbhar, Deepak, J, Tariq","roleAtSource":"Article author"},"source":{"title":"LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2605.16675","publishedAt":"2026-05-15","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors introduce LinAlg-Bench to evaluate 10 frontier large language models on structured linear algebra computation across 3x3, 4x4 and 5x5 matrices.","excerpt":"evaluating 10 frontier large language models on structured linear algebra computation across a strict dimensional gradient of 3x3, 4x4, and 5x5 matrices","url":"https://arxiv.org/abs/2605.16675","relationship":"Source-stated"},{"text":"The authors report a change in failure patterns: execution errors occur below 4x4 scale, while at larger scales models shift toward computational abandonment and fabricated responses.","excerpt":"below it, models fail through execution errors -- sign tracking failures, arithmetic drift, and parity errors; above it, failure transitions to computational abandonment","url":"https://arxiv.org/abs/2605.16675","relationship":"Source-stated"},{"text":"The authors say the fabrication-to-abandonment transition is near-universal across model tiers and architectures and suggest it reflects a working memory limit rather than a knowledge gap.","excerpt":"suggesting a working memory limit rather than a knowledge gap","url":"https://arxiv.org/abs/2605.16675","relationship":"Source-stated"},{"text":"The authors report that solution strategy rigidity is a near-perfect predictor of 5x5 determinant accuracy.","excerpt":"solution strategy rigidity is a near-perfect predictor of 5x5 determinant accuracy","url":"https://arxiv.org/abs/2605.16675","relationship":"Source-stated"},{"text":"The authors describe constraint-aware confabulation as a novel structured hallucination failure mode.","excerpt":"document constraint-aware confabulation as a novel structured hallucination failure mode","url":"https://arxiv.org/abs/2605.16675","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2605.16675","checkedAt":"2026-11-09T00:50:07.609Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What evidence in the paper supports the authors’ interpretation that the shift in failures reflects a working memory limit rather than a knowledge gap?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning","summary":"Shradha Agarwal, Deepak Rajbhar and Tariq J present LinAlg-Bench, a diagnostic benchmark of 10 frontier large language models solving structured linear algebra problems on 3x3, 4x4 and 5x5 matrices. The authors argue that the models’ errors follow a pattern rather than occurring at random: at 3x3, failures tend to involve execution errors such as sign tracking and arithmetic, while at 4x4 and 5x5, models often abandon computation and produce fabricated responses. They describe this shift as near-universal across model tiers and architectures, and suggest it reflects a working memory limit rather than a knowledge gap. The benchmark uses 660 SymPy-certified problems across 9 task types and analyzes 6,600 outputs; its forensic pipeline classifies failures into error tags and subtypes. The authors also report that solution strategy rigidity is a near-perfect predictor of 5x5 determinant accuracy, and identify constraint-aware confabulation as a structured hallucination failure mode. These findings make the benchmark a way to examine not just whether models get answers wrong, but how their failures change with problem scale. The claims concern this benchmark’s tasks and evaluated models; the supplied abstract does not establish how the findings extend beyond them.","question":"What evidence in the paper supports the authors’ interpretation that the shift in failures reflects a working memory limit rather than a knowledge gap?","claims":[{"text":"The authors introduce LinAlg-Bench to evaluate 10 frontier large language models on structured linear algebra computation across 3x3, 4x4 and 5x5 matrices.","excerpt":"evaluating 10 frontier large language models on structured linear algebra computation across a strict dimensional gradient of 3x3, 4x4, and 5x5 matrices","support":"source-stated"},{"text":"The authors report a change in failure patterns: execution errors occur below 4x4 scale, while at larger scales models shift toward computational abandonment and fabricated responses.","excerpt":"below it, models fail through execution errors -- sign tracking failures, arithmetic drift, and parity errors; above it, failure transitions to computational abandonment","support":"source-stated"},{"text":"The authors say the fabrication-to-abandonment transition is near-universal across model tiers and architectures and suggest it reflects a working memory limit rather than a knowledge gap.","excerpt":"suggesting a working memory limit rather than a knowledge gap","support":"source-stated"},{"text":"The authors report that solution strategy rigidity is a near-perfect predictor of 5x5 determinant accuracy.","excerpt":"solution strategy rigidity is a near-perfect predictor of 5x5 determinant accuracy","support":"source-stated"},{"text":"The authors describe constraint-aware confabulation as a novel structured hallucination failure mode.","excerpt":"document constraint-aware confabulation as a novel structured hallucination failure mode","support":"source-stated"}],"limitations":["The supplied evidence is the arXiv abstract, not the paper’s full methods or results; it does not provide the details needed to assess the benchmark construction or the reported analyses.","The findings described concern the evaluated models and the benchmark’s structured linear algebra tasks; the abstract does not establish whether the patterns extend to other tasks or settings."],"mediaId":"f8f2ffba619f685692def5cfc315279eb00983794d61332faf93c99696e1c9a9","correctionReason":null,"id":"84b37ec63bc097ea99cfc92d42494c2135a87ecaad3b29f9cbe50abaf12cc36e","publicationId":"8f726e9b-117f-43ae-8896-14577d0e7054","version":1,"sourceUrl":"https://arxiv.org/abs/2605.16675","sourceTitle":"LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-05-15","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-06T05:10:43.724Z","createdAt":"2026-11-09T00:18:21.904Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"f8f2ffba619f685692def5cfc315279eb00983794d61332faf93c99696e1c9a9","kind":"article","url":"https://arxiv.org/abs/2605.16675","sourceUrl":"https://arxiv.org/abs/2605.16675","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T00:17:36.231Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2605.16675","checkedAt":"2026-11-09T00:50:07.609Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"f8f2ffba619f685692def5cfc315279eb00983794d61332faf93c99696e1c9a9","confidence":0.88,"provider":"typesafe","model":"jev-1.13.0","inputHash":"3c4946ba8d8e2c8fb194731da8da30e7bae2bc453c3626f1be018bd1d3a5475e","decidedAt":"2026-11-09T00:17:37.862Z"},"provenance":{"textHash":"88f7b1bf4e51e2a7cc3841ae16056aba3e07e17ee22a6e789489934e0cb0c044","observationId":"b6612c31a31fa692ddc1ed6661a639feef7cd8acf8a2f3a516de45de85b8226a","retrievedAt":"2026-11-09T00:17:35.968Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"5cd69e47-1369-4f5d-82e2-71c5943034a1"},"reviewer":{"model":"gpt-6-luna","runId":"f2cd6fef-a57f-49bb-8421-26b64a3a2855"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T00:18:21.904Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Agarwal, Shradha","url":null,"kind":"unknown"},{"name":"Rajbhar, Deepak","url":null,"kind":"unknown"},{"name":"J, Tariq","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Agarwal, Shradha","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Agarwal, Shradha","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2605.16675","locator":"head/meta[name=author|citation_author]"},"assertionId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T00:50:07.609Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a","sourceUrl":"https://arxiv.org/abs/2605.16675","contentSha256":"64e9ca9a1d5efaeac17b3390a954b124ea75bf2dc789efff12972988cab426a7","textHash":"88f7b1bf4e51e2a7cc3841ae16056aba3e07e17ee22a6e789489934e0cb0c044","observedAt":"2026-11-09T00:50:07.609Z"}},{"actor":{"name":"Rajbhar, Deepak","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Rajbhar, Deepak","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2605.16675","locator":"head/meta[name=author|citation_author]"},"assertionId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T00:50:07.609Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a","sourceUrl":"https://arxiv.org/abs/2605.16675","contentSha256":"64e9ca9a1d5efaeac17b3390a954b124ea75bf2dc789efff12972988cab426a7","textHash":"88f7b1bf4e51e2a7cc3841ae16056aba3e07e17ee22a6e789489934e0cb0c044","observedAt":"2026-11-09T00:50:07.609Z"}},{"actor":{"name":"J, Tariq","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"J, Tariq","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2605.16675","locator":"head/meta[name=author|citation_author]"},"assertionId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T00:50:07.609Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a","sourceUrl":"https://arxiv.org/abs/2605.16675","contentSha256":"64e9ca9a1d5efaeac17b3390a954b124ea75bf2dc789efff12972988cab426a7","textHash":"88f7b1bf4e51e2a7cc3841ae16056aba3e07e17ee22a6e789489934e0cb0c044","observedAt":"2026-11-09T00:50:07.609Z"}}],"participationCoverage":{"total":3,"returned":3,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2605.16675","snapshotId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a","contentSha256":"64e9ca9a1d5efaeac17b3390a954b124ea75bf2dc789efff12972988cab426a7","textHash":"88f7b1bf4e51e2a7cc3841ae16056aba3e07e17ee22a6e789489934e0cb0c044","observedAt":"2026-11-09T00:50:07.609Z","basis":"publisher-page-metadata","sourceLinks":43,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/84b37ec63bc097ea99cfc92d42494c2135a87ecaad3b29f9cbe50abaf12cc36e/source"}},"canonicalUrl":"https://escapemeridian.com/readings/linalg-bench-a-forensic-benchmark-revealing-structural-failure-84b37ec6","formats":{"html":"/readings/linalg-bench-a-forensic-benchmark-revealing-structural-failure-84b37ec6","markdown":"/readings/linalg-bench-a-forensic-benchmark-revealing-structural-failure-84b37ec6.md","json":"/readings/linalg-bench-a-forensic-benchmark-revealing-structural-failure-84b37ec6.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"},{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"},{"id":"term-hallucination","name":"Hallucination","explanation":"When an AI system states something false or made up as if it were true.","referenceUrl":"https://developers.google.com/machine-learning/glossary#hallucination"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"84b37ec63bc097ea99cfc92d42494c2135a87ecaad3b29f9cbe50abaf12cc36e","revision":"8f726e9b-117f-43ae-8896-14577d0e7054","revisionKind":"publication-id","title":"LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning","url":"https://escapemeridian.com/readings/linalg-bench-a-forensic-benchmark-revealing-structural-failure-84b37ec6"},"source":{"type":"source","id":"https://arxiv.org/abs/2605.16675","revision":"88f7b1bf4e51e2a7cc3841ae16056aba3e07e17ee22a6e789489934e0cb0c044","revisionKind":"sha256-source-text","title":"LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning","url":"https://arxiv.org/abs/2605.16675"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"84b37ec63bc097ea99cfc92d42494c2135a87ecaad3b29f9cbe50abaf12cc36e","title":"LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning","argument":"Shradha Agarwal, Deepak Rajbhar and Tariq J present LinAlg-Bench, a diagnostic benchmark of 10 frontier large language models solving structured linear algebra problems on 3x3, 4x4 and 5x5 matrices. The authors argue that the models’ errors follow a pattern rather than occurring at random: at 3x3, failures tend to involve execution errors such as sign tracking and arithmetic, while at 4x4 and 5x5, models often abandon computation and produce fabricated responses. They describe this shift as near-universal across model tiers and architectures, and suggest it reflects a working memory limit rather than a knowledge gap. The benchmark uses 660 SymPy-certified problems across 9 task types and analyzes 6,600 outputs; its forensic pipeline classifies failures into error tags and subtypes. The authors also report that solution strategy rigidity is a near-perfect predictor of 5x5 determinant accuracy, and identify constraint-aware confabulation as a structured hallucination failure mode. These findings make the benchmark a way to examine not just whether models get answers wrong, but how their failures change with problem scale. The claims concern this benchmark’s tasks and evaluated models; the supplied abstract does not establish how the findings extend beyond them.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the arXiv abstract, not the paper’s full methods or results; it does not provide the details needed to assess the benchmark construction or the reported analyses. The findings described concern the evaluated models and the benchmark’s structured linear algebra tasks; the abstract does not establish whether the patterns extend to other tasks or settings.","speaker":{"name":"Agarwal, Shradha, Rajbhar, Deepak, J, Tariq","roleAtSource":"Article author"},"source":{"title":"LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2605.16675","publishedAt":"2026-05-15","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors introduce LinAlg-Bench to evaluate 10 frontier large language models on structured linear algebra computation across 3x3, 4x4 and 5x5 matrices.","excerpt":"evaluating 10 frontier large language models on structured linear algebra computation across a strict dimensional gradient of 3x3, 4x4, and 5x5 matrices","url":"https://arxiv.org/abs/2605.16675","relationship":"Source-stated"},{"text":"The authors report a change in failure patterns: execution errors occur below 4x4 scale, while at larger scales models shift toward computational abandonment and fabricated responses.","excerpt":"below it, models fail through execution errors -- sign tracking failures, arithmetic drift, and parity errors; above it, failure transitions to computational abandonment","url":"https://arxiv.org/abs/2605.16675","relationship":"Source-stated"},{"text":"The authors say the fabrication-to-abandonment transition is near-universal across model tiers and architectures and suggest it reflects a working memory limit rather than a knowledge gap.","excerpt":"suggesting a working memory limit rather than a knowledge gap","url":"https://arxiv.org/abs/2605.16675","relationship":"Source-stated"},{"text":"The authors report that solution strategy rigidity is a near-perfect predictor of 5x5 determinant accuracy.","excerpt":"solution strategy rigidity is a near-perfect predictor of 5x5 determinant accuracy","url":"https://arxiv.org/abs/2605.16675","relationship":"Source-stated"},{"text":"The authors describe constraint-aware confabulation as a novel structured hallucination failure mode.","excerpt":"document constraint-aware confabulation as a novel structured hallucination failure mode","url":"https://arxiv.org/abs/2605.16675","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2605.16675","checkedAt":"2026-11-09T00:50:07.609Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What evidence in the paper supports the authors’ interpretation that the shift in failures reflects a working memory limit rather than a knowledge gap?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning","summary":"Shradha Agarwal, Deepak Rajbhar and Tariq J present LinAlg-Bench, a diagnostic benchmark of 10 frontier large language models solving structured linear algebra problems on 3x3, 4x4 and 5x5 matrices. The authors argue that the models’ errors follow a pattern rather than occurring at random: at 3x3, failures tend to involve execution errors such as sign tracking and arithmetic, while at 4x4 and 5x5, models often abandon computation and produce fabricated responses. They describe this shift as near-universal across model tiers and architectures, and suggest it reflects a working memory limit rather than a knowledge gap. The benchmark uses 660 SymPy-certified problems across 9 task types and analyzes 6,600 outputs; its forensic pipeline classifies failures into error tags and subtypes. The authors also report that solution strategy rigidity is a near-perfect predictor of 5x5 determinant accuracy, and identify constraint-aware confabulation as a structured hallucination failure mode. These findings make the benchmark a way to examine not just whether models get answers wrong, but how their failures change with problem scale. The claims concern this benchmark’s tasks and evaluated models; the supplied abstract does not establish how the findings extend beyond them.","question":"What evidence in the paper supports the authors’ interpretation that the shift in failures reflects a working memory limit rather than a knowledge gap?","claims":[{"text":"The authors introduce LinAlg-Bench to evaluate 10 frontier large language models on structured linear algebra computation across 3x3, 4x4 and 5x5 matrices.","excerpt":"evaluating 10 frontier large language models on structured linear algebra computation across a strict dimensional gradient of 3x3, 4x4, and 5x5 matrices","support":"source-stated"},{"text":"The authors report a change in failure patterns: execution errors occur below 4x4 scale, while at larger scales models shift toward computational abandonment and fabricated responses.","excerpt":"below it, models fail through execution errors -- sign tracking failures, arithmetic drift, and parity errors; above it, failure transitions to computational abandonment","support":"source-stated"},{"text":"The authors say the fabrication-to-abandonment transition is near-universal across model tiers and architectures and suggest it reflects a working memory limit rather than a knowledge gap.","excerpt":"suggesting a working memory limit rather than a knowledge gap","support":"source-stated"},{"text":"The authors report that solution strategy rigidity is a near-perfect predictor of 5x5 determinant accuracy.","excerpt":"solution strategy rigidity is a near-perfect predictor of 5x5 determinant accuracy","support":"source-stated"},{"text":"The authors describe constraint-aware confabulation as a novel structured hallucination failure mode.","excerpt":"document constraint-aware confabulation as a novel structured hallucination failure mode","support":"source-stated"}],"limitations":["The supplied evidence is the arXiv abstract, not the paper’s full methods or results; it does not provide the details needed to assess the benchmark construction or the reported analyses.","The findings described concern the evaluated models and the benchmark’s structured linear algebra tasks; the abstract does not establish whether the patterns extend to other tasks or settings."],"mediaId":"f8f2ffba619f685692def5cfc315279eb00983794d61332faf93c99696e1c9a9","correctionReason":null,"id":"84b37ec63bc097ea99cfc92d42494c2135a87ecaad3b29f9cbe50abaf12cc36e","publicationId":"8f726e9b-117f-43ae-8896-14577d0e7054","version":1,"sourceUrl":"https://arxiv.org/abs/2605.16675","sourceTitle":"LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-05-15","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-06T05:10:43.724Z","createdAt":"2026-11-09T00:18:21.904Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"f8f2ffba619f685692def5cfc315279eb00983794d61332faf93c99696e1c9a9","kind":"article","url":"https://arxiv.org/abs/2605.16675","sourceUrl":"https://arxiv.org/abs/2605.16675","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T00:17:36.231Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2605.16675","checkedAt":"2026-11-09T00:50:07.609Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"f8f2ffba619f685692def5cfc315279eb00983794d61332faf93c99696e1c9a9","confidence":0.88,"provider":"typesafe","model":"jev-1.13.0","inputHash":"3c4946ba8d8e2c8fb194731da8da30e7bae2bc453c3626f1be018bd1d3a5475e","decidedAt":"2026-11-09T00:17:37.862Z"},"provenance":{"textHash":"88f7b1bf4e51e2a7cc3841ae16056aba3e07e17ee22a6e789489934e0cb0c044","observationId":"b6612c31a31fa692ddc1ed6661a639feef7cd8acf8a2f3a516de45de85b8226a","retrievedAt":"2026-11-09T00:17:35.968Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"5cd69e47-1369-4f5d-82e2-71c5943034a1"},"reviewer":{"model":"gpt-6-luna","runId":"f2cd6fef-a57f-49bb-8421-26b64a3a2855"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T00:18:21.904Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Agarwal, Shradha","url":null,"kind":"unknown"},{"name":"Rajbhar, Deepak","url":null,"kind":"unknown"},{"name":"J, Tariq","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Agarwal, Shradha","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Agarwal, Shradha","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2605.16675","locator":"head/meta[name=author|citation_author]"},"assertionId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T00:50:07.609Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a","sourceUrl":"https://arxiv.org/abs/2605.16675","contentSha256":"64e9ca9a1d5efaeac17b3390a954b124ea75bf2dc789efff12972988cab426a7","textHash":"88f7b1bf4e51e2a7cc3841ae16056aba3e07e17ee22a6e789489934e0cb0c044","observedAt":"2026-11-09T00:50:07.609Z"}},{"actor":{"name":"Rajbhar, Deepak","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Rajbhar, Deepak","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2605.16675","locator":"head/meta[name=author|citation_author]"},"assertionId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T00:50:07.609Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a","sourceUrl":"https://arxiv.org/abs/2605.16675","contentSha256":"64e9ca9a1d5efaeac17b3390a954b124ea75bf2dc789efff12972988cab426a7","textHash":"88f7b1bf4e51e2a7cc3841ae16056aba3e07e17ee22a6e789489934e0cb0c044","observedAt":"2026-11-09T00:50:07.609Z"}},{"actor":{"name":"J, Tariq","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"J, Tariq","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2605.16675","locator":"head/meta[name=author|citation_author]"},"assertionId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T00:50:07.609Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a","sourceUrl":"https://arxiv.org/abs/2605.16675","contentSha256":"64e9ca9a1d5efaeac17b3390a954b124ea75bf2dc789efff12972988cab426a7","textHash":"88f7b1bf4e51e2a7cc3841ae16056aba3e07e17ee22a6e789489934e0cb0c044","observedAt":"2026-11-09T00:50:07.609Z"}}],"participationCoverage":{"total":3,"returned":3,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2605.16675","snapshotId":"0ef04c944a7c902aff0661728a7f747f405269377b625324c4f4122827d6539a","contentSha256":"64e9ca9a1d5efaeac17b3390a954b124ea75bf2dc789efff12972988cab426a7","textHash":"88f7b1bf4e51e2a7cc3841ae16056aba3e07e17ee22a6e789489934e0cb0c044","observedAt":"2026-11-09T00:50:07.609Z","basis":"publisher-page-metadata","sourceLinks":43,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/84b37ec63bc097ea99cfc92d42494c2135a87ecaad3b29f9cbe50abaf12cc36e/source"}}}],"audits":[{"publicationId":"8f726e9b-117f-43ae-8896-14577d0e7054","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading correctly credits Shradha Agarwal, Deepak Rajbhar and Tariq J and frames the findings and interpretations as the authors' claims rather than established facts.","excerpt":"Authors:Shradha Agarwal, Deepak Rajbhar, Tariq J"},{"dimension":"summary","status":"uncertain","explanation":"The summary assigns 4x4 to the abandonment regime ('at 4x4 and 5x5, models often abandon computation'). The abstract places the threshold at 4x4, with execution errors below it and abandonment above it, so 4x4's side is ambiguous. Smallest repair: say failures shift at a threshold around 4x4. Central terms such as SymPy-certified, determinant, frontier models and confabulation are also left unexplained for non-specialist readers. Term-library tooltips or a brief plain gloss would fix this.","excerpt":"a sharp behavioral threshold at 4x4 scale: below it, models fail through execution errors"},{"dimension":"date","status":"pass","explanation":"The originalPublished date of 2026-05-15 matches the v1 submission. It correctly uses the original submission rather than the 4 Nov 2026 revision or the feed date.","excerpt":"Submitted on 15 May 2026 (v1), last revised 4 Nov 2026 (this version, v2)"},{"dimension":"support","status":"pass","explanation":"The claims and their excerpts match the abstract. The working-memory interpretation is kept as the authors' suggestion. The limitations accurately note that the evidence is abstract-only and that scope extends only to these tasks. The question follows from the evidence without leading.","excerpt":"suggesting a working memory limit rather than a knowledge gap"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper, which is the original source.","excerpt":null}],"createdAt":"2026-11-09T00:18:21.904Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}