arXiv paper introduces LISA for long-context reasoning

arXiv paper introduces LISA for long-context reasoning

The authors introduce LISA, a replacement for self-attention that combines a linear-attention branch for long-range memory with a branch that selects relevant tokens for sparse self-attention. A learned gate combines the branches. The authors report that, on DeepSeek-distilled Qwen models, LISA cuts measured eager-mode mean sample latency by 23.9% at 16K context without degrading long-context reasoning quality. They also report a 4.2 percentage-point improvement in the equal-weight average across nine mathematical and code reasoning benchmarks for the gated LISA-1.5B model, compared with the full-attention baseline. These are reported experimental results; the abstract does not establish performance in production or on other models. How well do the reported latency and reasoning results hold beyond the tested models and benchmarks?

Publication: 3a8a24fb-10bd-4930-8a80-780540f0db8f

attribution — pass: The draft consistently credits the paper's authors on arXiv for LISA and its reported results. It presents the latency and benchmark figures as the authors' reports, not as established facts.

We therefore introduce LISA (Linear-Indexed Sparse Attention), a plug-and-play replacement for self-attention

summary — uncertain: The draft leaves out the paper's central motivation. Standard self-attention costs grow quadratically with context length, which makes long chain-of-thought deployment expensive. Without that, a general reader cannot see why replacing self-attention matters. Central terms such as self-attention, linear attention, sparse self-attention, tokens, eager-mode and 16K context are not explained in plain language or flagged for tooltips. The draft also drops the source's point that LISA needs no full pretraining. Smallest repair: add one sentence on the quadratic-cost problem and the no-pretraining point, and gloss the key terms.

standard self-attention incurs quadratic cost in context length, making long-CoT deployment costly in production

date — pass: originalPublished is 2026-05-29. This matches the v1 submission date. The October dates belong to the v2 revision and the feed entry, not the first publication.

Submitted on 29 May 2026 (v1), last revised 27 Oct 2026 (this version, v2)

support — pass: All three claims match the abstract verbatim, including scope and conditions: eager-mode, mean sample latency, 16K context, DeepSeek-distilled Qwen models, gated LISA-1.5B, and the equal-weight average over nine benchmarks. The limitations and the question are appropriately conditional and do not invent benefits or costs.

the gated LISA-1.5B model improves the equal-weight benchmark average by 4.2 percentage points relative to the full-attention baseline

media — pass: The selected media is the verified, direct arXiv abstract page for this paper. That is the most direct way to reach the source.