Shi, Chenglong, Yuan, Zekai, Song, Yue, Pu, Jon, Wu, Will · Oct 28, 2026
SpaceAct: Benchmarking 3D Spatial and Temporal Reasoning via Closed-Loop Action and Active Perception
Markdown · JSON · Original source
The authors introduce SpaceAct, an interactive for testing whether (MLLMs) can reason about 3D space through physical action, rather than only describe images or plan symbolically. Its tasks use interlocking bricks and mechanical puzzles, with models able to rotate a camera, inspect rulers, issue 6-DoF poses, and adjust actions using feedback. The benchmark is designed to test whether models can handle occlusion, collision, statics, and timing without privileged coordinates. Across 10 frontier MLLMs on the 44 hardest instances, scores ranged from 4.8 to 79.1. The authors report that adding a reference plan helped some models, but collisions remained; providing exact 3D state improved scores and cut collisions in the reported comparisons. They identify collision or observation loops as 46% of failures. Their central argument is that a valid plan is not enough: visual grounding and action refinement remain bottlenecks in these tasks. SpaceAct offers a testbed for evaluating active multi-view exploration and explicit 3D grounding, but the abstract does not establish how performance transfers beyond its benchmark. How well would performance on SpaceAct predict MLLM behavior in physical settings beyond these puzzles?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 28, 2026 (2026-10-28).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-13T04:17:21.982Z. First observed: 2026-10-30T06:40:25.159Z. Retrieved: 2026-10-31T08:16:51.562Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The supplied evidence is the abstract, so it does not provide details needed to assess the design or reported comparisons beyond the summary figures. The abstract reports benchmark results but does not establish whether performance predicts behavior in physical settings beyond the benchmark's interlocking-brick and mechanical-puzzle tasks.
Open question
How well would performance on SpaceAct predict MLLM behavior in physical settings beyond these puzzles?
Source-stated
The authors present SpaceAct as an interactive benchmark for evaluating 3D spatiotemporal reasoning through physical action and active perception.
We introduce SpaceAct, an interactive benchmark evaluating 3D spatiotemporal reasoning via closed-loop physical action and active perception
Source-stated
The benchmark contains 108 verified instances across 36 task families in six categories, without privileged coordinates.
it has 108 verified instances across 36 task families in six categories, without privileged coordinates
Source-stated
On the 44 hardest instances, scores across 10 frontier MLLMs ranged from 4.8 to 79.1.
Across 10 frontier MLLMs on the 44 hardest instances, scores span 4.8-79.1
Source-stated
For the top three models, adding a reference plan closed 44% of the gap but left collisions at 5.2 per episode; adding exact 3D state closed 90% and reduced collisions to 0.5.
for the top three, a reference plan (+plan) closes 44% of the gap while leaving collisions at 5.2 per episode, whereas adding exact 3D state (+state) closes 90% and cuts collisions to 0.5
Source-stated
The authors identify 3D visual grounding and action refinement as bottlenecks, even when a valid plan is available.
showing that even with a valid plan, MLLMs remain bottlenecked by 3D visual grounding and action refinement
Source and contributor context
Shi, Chenglong, Yuan, Zekai, Song, Yue, Pu, Jon, Wu, Will · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Multimodal model
A model that works with more than one kind of input or output, such as text, images and audio.
Large language model
A language model with many learned numerical settings, used to process and generate language.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 6a040580-cb78-4ab7-8530-ca370acdfdb1.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-13T04:17:21.982Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 65d82fb5517c5e8660271fd8f432a4a59afe47d24a669578b1f48586d23e3a71. Source text hash: 74dd5d5028e767ae2193c744e2c361c88a0740c8a85173f1bc47c9696116f98f.
Revision and source-check history
Version 1 · 2026-11-13T04:17:21.982Z
SpaceAct: Benchmarking 3D Spatial and Temporal Reasoning via Closed-Loop Action and Active Perception
The authors introduce SpaceAct, an interactive for testing whether (MLLMs) can reason about 3D space through physical action, rather than only describe images or plan symbolically. Its tasks use interlocking bricks and mechanical puzzles, with models able to rotate a camera, inspect rulers, issue 6-DoF poses, and adjust actions using feedback. The benchmark is designed to test whether models can handle occlusion, collision, statics, and timing without privileged coordinates. Across 10 frontier MLLMs on the 44 hardest instances, scores ranged from 4.8 to 79.1. The authors report that adding a reference plan helped some models, but collisions remained; providing exact 3D state improved scores and cut collisions in the reported comparisons. They identify collision or observation loops as 46% of failures. Their central argument is that a valid plan is not enough: visual grounding and action refinement remain bottlenecks in these tasks. SpaceAct offers a testbed for evaluating active multi-view exploration and explicit 3D grounding, but the abstract does not establish how performance transfers beyond its benchmark. How well would performance on SpaceAct predict MLLM behavior in physical settings beyond these puzzles?
Original source
cs.AI · Article
Source details
Original source date: Oct 28, 2026 (2026-10-28).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-13T04:17:21.982Z. First observed: 2026-10-30T06:40:25.159Z. Retrieved: 2026-10-31T08:16:51.562Z.
Limits
The supplied evidence is the abstract, so it does not provide details needed to assess the design or reported comparisons beyond the summary figures. The abstract reports benchmark results but does not establish whether performance predicts behavior in physical settings beyond the benchmark's interlocking-brick and mechanical-puzzle tasks.
Open question
How well would performance on SpaceAct predict MLLM behavior in physical settings beyond these puzzles?
Source-stated
The authors present SpaceAct as an interactive benchmark for evaluating 3D spatiotemporal reasoning through physical action and active perception.
We introduce SpaceAct, an interactive benchmark evaluating 3D spatiotemporal reasoning via closed-loop physical action and active perception
Source-stated
The benchmark contains 108 verified instances across 36 task families in six categories, without privileged coordinates.
it has 108 verified instances across 36 task families in six categories, without privileged coordinates
Source-stated
On the 44 hardest instances, scores across 10 frontier MLLMs ranged from 4.8 to 79.1.
Across 10 frontier MLLMs on the 44 hardest instances, scores span 4.8-79.1
Source-stated
For the top three models, adding a reference plan closed 44% of the gap but left collisions at 5.2 per episode; adding exact 3D state closed 90% and reduced collisions to 0.5.
for the top three, a reference plan (+plan) closes 44% of the gap while leaving collisions at 5.2 per episode, whereas adding exact 3D state (+state) closes 90% and cuts collisions to 0.5
Source-stated
The authors identify 3D visual grounding and action refinement as bottlenecks, even when a valid plan is available.
showing that even with a valid plan, MLLMs remain bottlenecked by 3D visual grounding and action refinement
Source and contributor context
Shi, Chenglong, Yuan, Zekai, Song, Yue, Pu, Jon, Wu, Will · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Multimodal model
A model that works with more than one kind of input or output, such as text, images and audio.
Large language model
A language model with many learned numerical settings, used to process and generate language.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 6a040580-cb78-4ab7-8530-ca370acdfdb1.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-13T04:17:21.982Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 65d82fb5517c5e8660271fd8f432a4a59afe47d24a669578b1f48586d23e3a71. Source text hash: 74dd5d5028e767ae2193c744e2c361c88a0740c8a85173f1bc47c9696116f98f.
passed · 2026-11-13T04:17:21.982Z · anthropic/claude-opus-5.5
Publication: 6a040580-cb78-4ab7-8530-ca370acdfdb1
attribution — pass: The reading attributes SpaceAct, the results and the bottleneck conclusion to the paper's authors. Its framing as an arXiv abstract matches the evidence, and the authors' interpretation is not presented as established fact.
Authors:Chenglong Shi, Zekai Yuan, Yue Song, Jon Pu, Will Wu
summary — pass: The reading leads with what the authors introduce, explains MLLMs and how the benchmark differs from describing images or planning symbolically, and conveys the central argument that a valid plan is not enough. Terms such as 6-DoF, privileged coordinates and statics are kept exactly, consistent with tooltip handling. The question follows from the evidence and is not leading. Minor nit: the closing testbed sentence could add 'the authors say', but the meaning is faithful.
showing that even with a valid plan, MLLMs remain bottlenecked by 3D visual grounding and action refinement
date — pass: The originalPublished date of 2026-10-28 matches the arXiv v1 submission date. The later feed date of Oct 31 is correctly not used.
[Submitted on 28 Oct 2026]
support — pass: Each claim excerpt is verbatim from the abstract. The summary's statements about plan-only gains with remaining collisions, state-based gains, and the 46% failure share match the source. The limitations correctly note that transfer beyond the benchmark is unestablished, and no implied benefits or costs are invented.
and 46% of failures are collision or observation loops
media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA model that works with more than one kind of input or output, such as text, images and audio.
Definition referenceA language model with many learned numerical settings, used to process and generate language.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA model that works with more than one kind of input or output, such as text, images and audio.
Definition referenceA language model with many learned numerical settings, used to process and generate language.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference