# arXiv paper introduces LISA for long-context reasoning [Canonical reading](https://escapemeridian.com/readings/arxiv-paper-introduces-lisa-for-long-context-reasoning-941bef05) The authors introduce LISA, a replacement for self-attention that combines a linear-attention branch for long-range memory with a branch that selects relevant tokens for sparse self-attention. A learned gate combines the branches. The authors report that, on DeepSeek-distilled Qwen models, LISA cuts measured eager-mode mean sample latency by 23.9% at 16K context without degrading long-context reasoning quality. They also report a 4.2 percentage-point improvement in the equal-weight average across nine mathematical and code reasoning benchmarks for the gated LISA-1.5B model, compared with the full-attention baseline. These are reported experimental results; the abstract does not establish performance in production or on other models. How well do the reported latency and reasoning results hold beyond the tested models and benchmarks? Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: May 29, 2026 (2026-05-29). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-13T23:31:21.964Z. First observed: 2026-10-28T04:34:38.228Z. Retrieved: 2026-10-30T04:02:49.502Z. - [LISA: Linear-Indexed Sparse Attention for Efficient Long-Context Reasoning](https://arxiv.org/abs/2607.19358) ## Limits The reported results concern experiments on DeepSeek-distilled Qwen models; the abstract does not establish performance on other models or in production. The supplied evidence is the paper's abstract and does not provide benchmark details, evaluation methods, or further results for independent assessment. ## Open question How well do the reported latency and reasoning results hold beyond the tested models and benchmarks? ## Source-stated The authors present LISA as a plug-and-play replacement for self-attention, combining linear attention with token selection for sparse self-attention. > We therefore introduce LISA (Linear-Indexed Sparse Attention), a plug-and-play replacement for self-attention - [Supporting source](https://arxiv.org/abs/2607.19358) ## Source-stated The authors report a 23.9% reduction in measured eager-mode mean sample latency at 16K context, without degradation in long-context reasoning quality, on DeepSeek-distilled Qwen models. > cuts measured eager-mode mean sample latency by 23.9% at 16K context without degrading long-context reasoning quality - [Supporting source](https://arxiv.org/abs/2607.19358) ## Source-stated The authors report that gated LISA-1.5B improves the equal-weight average by 4.2 percentage points over the full-attention baseline across nine mathematical and code reasoning benchmarks. > Across nine mathematical and code reasoning benchmarks, the gated LISA-1.5B model improves the equal-weight benchmark average by 4.2 percentage points relative to the full-attention baseline. - [Supporting source](https://arxiv.org/abs/2607.19358) ## Source and contributor context Zhao, Yu, Zhang, Zekun, Jiang, Fan, Zeng, Bo, Wang, Xintong, Liu, Chonghan, Shan, Shimin, Du, Yichao, Liu, Yu · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Latency The delay between sending a request and receiving the result. - [Definition reference](https://developers.google.com/machine-learning/glossary#latency) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 3a8a24fb-10bd-4930-8a80-780540f0db8f. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T23:31:21.964Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 6cedf7c31f267ac1afc5b93a5101b1f71c995e43d46b192aaea47039e735c6e1. Source text hash: 8e6d441ad101a9f17c513fa3331ff7f423a1ffcdbbef9d802d5bb9a23313616b. ## Revision and source-check history ### Version 1 — 2026-11-13T23:31:21.964Z The authors introduce LISA, a replacement for self-attention that combines a linear-attention branch for long-range memory with a branch that selects relevant tokens for sparse self-attention. A learned gate combines the branches. The authors report that, on DeepSeek-distilled Qwen models, LISA cuts measured eager-mode mean sample latency by 23.9% at 16K context without degrading long-context reasoning quality. They also report a 4.2 percentage-point improvement in the equal-weight average across nine mathematical and code reasoning benchmarks for the gated LISA-1.5B model, compared with the full-attention baseline. These are reported experimental results; the abstract does not establish performance in production or on other models. How well do the reported latency and reasoning results hold beyond the tested models and benchmarks? ## Original source arXiv cs.AI · Article Original source date: May 29, 2026 (2026-05-29). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-13T23:31:21.964Z. First observed: 2026-10-28T04:34:38.228Z. Retrieved: 2026-10-30T04:02:49.502Z. - [LISA: Linear-Indexed Sparse Attention for Efficient Long-Context Reasoning](https://arxiv.org/abs/2607.19358) ## Limits The reported results concern experiments on DeepSeek-distilled Qwen models; the abstract does not establish performance on other models or in production. The supplied evidence is the paper's abstract and does not provide benchmark details, evaluation methods, or further results for independent assessment. ## Open question How well do the reported latency and reasoning results hold beyond the tested models and benchmarks? ## Source-stated The authors present LISA as a plug-and-play replacement for self-attention, combining linear attention with token selection for sparse self-attention. > We therefore introduce LISA (Linear-Indexed Sparse Attention), a plug-and-play replacement for self-attention - [Supporting source](https://arxiv.org/abs/2607.19358) ## Source-stated The authors report a 23.9% reduction in measured eager-mode mean sample latency at 16K context, without degradation in long-context reasoning quality, on DeepSeek-distilled Qwen models. > cuts measured eager-mode mean sample latency by 23.9% at 16K context without degrading long-context reasoning quality - [Supporting source](https://arxiv.org/abs/2607.19358) ## Source-stated The authors report that gated LISA-1.5B improves the equal-weight average by 4.2 percentage points over the full-attention baseline across nine mathematical and code reasoning benchmarks. > Across nine mathematical and code reasoning benchmarks, the gated LISA-1.5B model improves the equal-weight benchmark average by 4.2 percentage points relative to the full-attention baseline. - [Supporting source](https://arxiv.org/abs/2607.19358) ## Source and contributor context Zhao, Yu, Zhang, Zekun, Jiang, Fan, Zeng, Bo, Wang, Xintong, Liu, Chonghan, Shan, Shimin, Du, Yichao, Liu, Yu · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Latency The delay between sending a request and receiving the result. - [Definition reference](https://developers.google.com/machine-learning/glossary#latency) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 3a8a24fb-10bd-4930-8a80-780540f0db8f. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T23:31:21.964Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 6cedf7c31f267ac1afc5b93a5101b1f71c995e43d46b192aaea47039e735c6e1. Source text hash: 8e6d441ad101a9f17c513fa3331ff7f423a1ffcdbbef9d802d5bb9a23313616b. ### uncertain — 2026-11-13T23:31:21.964Z — anthropic/claude-opus-5.5 Publication: 3a8a24fb-10bd-4930-8a80-780540f0db8f attribution — pass: The draft consistently credits the paper's authors on arXiv for LISA and its reported results. It presents the latency and benchmark figures as the authors' reports, not as established facts. > We therefore introduce LISA (Linear-Indexed Sparse Attention), a plug-and-play replacement for self-attention summary — uncertain: The draft leaves out the paper's central motivation. Standard self-attention costs grow quadratically with context length, which makes long chain-of-thought deployment expensive. Without that, a general reader cannot see why replacing self-attention matters. Central terms such as self-attention, linear attention, sparse self-attention, tokens, eager-mode and 16K context are not explained in plain language or flagged for tooltips. The draft also drops the source's point that LISA needs no full pretraining. Smallest repair: add one sentence on the quadratic-cost problem and the no-pretraining point, and gloss the key terms. > standard self-attention incurs quadratic cost in context length, making long-CoT deployment costly in production date — pass: originalPublished is 2026-05-29. This matches the v1 submission date. The October dates belong to the v2 revision and the feed entry, not the first publication. > Submitted on 29 May 2026 (v1), last revised 27 Oct 2026 (this version, v2) support — pass: All three claims match the abstract verbatim, including scope and conditions: eager-mode, mean sample latency, 16K context, DeepSeek-distilled Qwen models, gated LISA-1.5B, and the equal-weight average over nine benchmarks. The limitations and the question are appropriately conditional and do not invent benefits or costs. > the gated LISA-1.5B model improves the equal-weight benchmark average by 4.2 percentage points relative to the full-attention baseline media — pass: The selected media is the verified, direct arXiv abstract page for this paper. That is the most direct way to reach the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/941bef057c296a979e07dcefda5c0848ddc55f61ccb83f756186bd5391c3ad11/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-paper-introduces-lisa-for-long-context-reasoning-941bef05.json)