Khan, Amil, Subrahmanya, Anantajit, Manjunath, B. S. · Oct 28, 2026
arXiv cs.AI: Geometry Matters in Packed 3D Attention
Markdown · JSON · Original source
Amil Khan, Anantajit Subrahmanya and B.S. Manjunath argue that packing a 3D volume into a token sequence can make sequence position reflect memory layout rather than the geometry of the sample. They introduce PACK3D, which uses exact local neighborhood attention, compressed summaries of distant context and axial RoPE on true token-grid coordinates. In a synthetic task where only token memory order changes, they report that learned positional and packed-sequence RoPE fail after repacking, while axial grid-coordinate RoPE stays exact. On Allen WTC-11 hiPSC microscopy, they report gains over a parameter-matched 3D U-Net on the 15-structure joint task and over a matched learned-position local-global baseline. They also report that a fused Triton implementation matches windowed-attention at 65k tokens while supplying longer-range context. Escape Meridian interpretation: the authors’ central point is that preserving spatial coordinates through packing may help attention use geometric structure rather than memorize a particular token order. How well does PACK3D perform beyond the reported synthetic task and microscopy comparisons?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 28, 2026 (2026-10-28).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-13T03:16:49.718Z. First observed: 2026-10-30T06:42:51.478Z. Retrieved: 2026-10-31T11:01:30.543Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The supplied evidence is the paper abstract, not its full methods or results; it does not provide numerical performance measures or detailed evaluation conditions. The reported synthetic task and Allen WTC-11 hiPSC microscopy comparisons do not establish performance on other datasets or applications.
Open question
How well does PACK3D perform beyond the reported synthetic task and microscopy comparisons?
Source-stated
The authors argue that packing a 3D volume into a token sequence can cause sequence-indexed positional mechanisms to learn memory layout rather than object geometry.
a positional mechanism indexed by sequence position can learn the memory layout rather than the object.
Source-stated
PACK3D uses exact local neighborhood attention, compresses distant context into contiguous 3D block summaries, and applies axial RoPE using true token-grid coordinates.
PACK3D keeps local neighborhood attention exact, compresses only distant context into contiguous 3D block summaries
Source-stated
In a controlled synthetic task that changes token memory order while holding the physical labeling rule fixed, the authors report that axial grid-coordinate RoPE stays exact while the compared positional methods collapse under repacking.
while axial grid-coordinate RoPE stays exact throughout.
Source-stated
On Allen WTC-11 hiPSC microscopy, the authors report that PACK3D improves over a parameter-matched 3D U-Net on the 15-structure joint task and over a matched learned-position local-global baseline.
on Allen WTC-11 hiPSC microscopy, PACK3D improves over a parameter-matched 3D U-Net on the 15-structure joint task
Source-stated
The authors report that a fused Triton implementation matches windowed-attention latency at 65k tokens while providing context beyond what windowing supplies.
matches windowed-attention latency at 65k tokens while supplying the long-range context windowing cannot
Escape Meridian interpretation
The authors’ results suggest that preserving spatial coordinates through packing can help attention use geometric structure rather than memorize token order.
the operator is not only correct but trainable at scale.
Source and contributor context
Khan, Amil, Subrahmanya, Anantajit, Manjunath, B. S. · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Embedding vector
A list of numbers that represents a piece of text, an image or another item so software can compare meanings.
Latency
The delay between sending a request and receiving the result.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: b4529309-a32b-4ff9-8b0c-e8dda2c96212.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T03:16:49.718Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 4bffbddb1c5fd245e152841f9b535030dece295da77e4ce110d01e271dd91682. Source text hash: b201456214245cd87d6f3ea618ee77e15c4b712cb1121491ececa9f8f0952520.
Revision and source-check history
Version 1 · 2026-11-13T03:16:49.718Z
arXiv cs.AI: Geometry Matters in Packed 3D Attention
Amil Khan, Anantajit Subrahmanya and B.S. Manjunath argue that packing a 3D volume into a token sequence can make sequence position reflect memory layout rather than the geometry of the sample. They introduce PACK3D, which uses exact local neighborhood attention, compressed summaries of distant context and axial RoPE on true token-grid coordinates. In a synthetic task where only token memory order changes, they report that learned positional and packed-sequence RoPE fail after repacking, while axial grid-coordinate RoPE stays exact. On Allen WTC-11 hiPSC microscopy, they report gains over a parameter-matched 3D U-Net on the 15-structure joint task and over a matched learned-position local-global baseline. They also report that a fused Triton implementation matches windowed-attention at 65k tokens while supplying longer-range context. Escape Meridian interpretation: the authors’ central point is that preserving spatial coordinates through packing may help attention use geometric structure rather than memorize a particular token order. How well does PACK3D perform beyond the reported synthetic task and microscopy comparisons?
Original source
cs.AI · Article
Source details
Original source date: Oct 28, 2026 (2026-10-28).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-13T03:16:49.718Z. First observed: 2026-10-30T06:42:51.478Z. Retrieved: 2026-10-31T11:01:30.543Z.
Limits
The supplied evidence is the paper abstract, not its full methods or results; it does not provide numerical performance measures or detailed evaluation conditions. The reported synthetic task and Allen WTC-11 hiPSC microscopy comparisons do not establish performance on other datasets or applications.
Open question
How well does PACK3D perform beyond the reported synthetic task and microscopy comparisons?
Source-stated
The authors argue that packing a 3D volume into a token sequence can cause sequence-indexed positional mechanisms to learn memory layout rather than object geometry.
a positional mechanism indexed by sequence position can learn the memory layout rather than the object.
Source-stated
PACK3D uses exact local neighborhood attention, compresses distant context into contiguous 3D block summaries, and applies axial RoPE using true token-grid coordinates.
PACK3D keeps local neighborhood attention exact, compresses only distant context into contiguous 3D block summaries
Source-stated
In a controlled synthetic task that changes token memory order while holding the physical labeling rule fixed, the authors report that axial grid-coordinate RoPE stays exact while the compared positional methods collapse under repacking.
while axial grid-coordinate RoPE stays exact throughout.
Source-stated
On Allen WTC-11 hiPSC microscopy, the authors report that PACK3D improves over a parameter-matched 3D U-Net on the 15-structure joint task and over a matched learned-position local-global baseline.
on Allen WTC-11 hiPSC microscopy, PACK3D improves over a parameter-matched 3D U-Net on the 15-structure joint task
Source-stated
The authors report that a fused Triton implementation matches windowed-attention latency at 65k tokens while providing context beyond what windowing supplies.
matches windowed-attention latency at 65k tokens while supplying the long-range context windowing cannot
Escape Meridian interpretation
The authors’ results suggest that preserving spatial coordinates through packing can help attention use geometric structure rather than memorize token order.
the operator is not only correct but trainable at scale.
Source and contributor context
Khan, Amil, Subrahmanya, Anantajit, Manjunath, B. S. · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Embedding vector
A list of numbers that represents a piece of text, an image or another item so software can compare meanings.
Latency
The delay between sending a request and receiving the result.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: b4529309-a32b-4ff9-8b0c-e8dda2c96212.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T03:16:49.718Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 4bffbddb1c5fd245e152841f9b535030dece295da77e4ce110d01e271dd91682. Source text hash: b201456214245cd87d6f3ea618ee77e15c4b712cb1121491ececa9f8f0952520.
uncertain · 2026-11-13T03:16:49.718Z · anthropic/claude-opus-5.5
Publication: b4529309-a32b-4ff9-8b0c-e8dda2c96212
attribution — pass: The paper is correctly attributed to Amil Khan, Anantajit Subrahmanya and B.S. Manjunath. Results are framed as what the authors argue or report, not as established fact.
Authors:Amil Khan, Anantajit Subrahmanya, B.S. Manjunath
summary — uncertain: The reading is faithful but hard to follow without specialist background. Central terms are never explained in plain language or flagged for tooltips: token sequence, attention, axial RoPE, positional embeddings, 3D U-Net, Triton, windowed attention and hiPSC microscopy. It also never says what 3D volumes are used for, such as cell microscopy imagery, or why a model learning memory layout instead of the object matters. Smallest repair: add one plain sentence on the failure (the model learns how data is stored, not the shape it represents) and add tooltips for RoPE, U-Net and hiPSC. The authors' controls attributing the gain to the coordinate system are also omitted.
Once the volume is packed, sequence order no longer encodes the geometry of the sample
date — pass: The originalPublished value of 2026-10-28 matches the v1 submission date on arXiv. The feed date of 2026-10-31 is later and correctly not used.
[Submitted on 28 Oct 2026]
support — uncertain: The source-stated claims match the abstract. However, the labeled interpretation (preserving coordinates helps attention use geometry rather than memorize token order) carries an excerpt about trainability at scale, which does not support it. Better support exists in the source, such as the spatial-interpretability sentence or the controls isolating the coordinate system. Smallest repair: swap the excerpt for one of these. The limitations and the question are appropriately cautious and add no unsupported benefits.
compressed global context therefore stays spatially interpretable after packing, instead of collapsing to anonymous pooled content
media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A list of numbers that represents a piece of text, an image or another item so software can compare meanings.
Definition referenceThe delay between sending a request and receiving the result.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA list of numbers that represents a piece of text, an image or another item so software can compare meanings.
Definition referenceThe delay between sending a request and receiving the result.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv