# SpaceAct: Benchmarking 3D Spatial and Temporal Reasoning via Closed-Loop Action and Active Perception [Canonical reading](https://escapemeridian.com/readings/spaceact-benchmarking-3d-spatial-and-temporal-reasoning-via-8de72948) The authors introduce SpaceAct, an interactive benchmark for testing whether multimodal large language models (MLLMs) can reason about 3D space through physical action, rather than only describe images or plan symbolically. Its tasks use interlocking bricks and mechanical puzzles, with models able to rotate a camera, inspect rulers, issue 6-DoF poses, and adjust actions using feedback. The benchmark is designed to test whether models can handle occlusion, collision, statics, and timing without privileged coordinates. Across 10 frontier MLLMs on the 44 hardest instances, scores ranged from 4.8 to 79.1. The authors report that adding a reference plan helped some models, but collisions remained; providing exact 3D state improved scores and cut collisions in the reported comparisons. They identify collision or observation loops as 46% of failures. Their central argument is that a valid plan is not enough: visual grounding and action refinement remain bottlenecks in these tasks. SpaceAct offers a testbed for evaluating active multi-view exploration and explicit 3D grounding, but the abstract does not establish how performance transfers beyond its benchmark. How well would performance on SpaceAct predict MLLM behavior in physical settings beyond these puzzles? Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Oct 28, 2026 (2026-10-28). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-13T04:17:21.982Z. First observed: 2026-10-30T06:40:25.159Z. Retrieved: 2026-10-31T08:16:51.562Z. - [SpaceAct: Benchmarking 3D Spatial and Temporal Reasoning via Closed-Loop Action and Active Perception](https://arxiv.org/abs/2610.38405) ## Limits The supplied evidence is the arXiv abstract, so it does not provide details needed to assess the benchmark design or reported comparisons beyond the summary figures. The abstract reports benchmark results but does not establish whether performance predicts behavior in physical settings beyond the benchmark's interlocking-brick and mechanical-puzzle tasks. ## Open question How well would performance on SpaceAct predict MLLM behavior in physical settings beyond these puzzles? ## Source-stated The authors present SpaceAct as an interactive benchmark for evaluating 3D spatiotemporal reasoning through physical action and active perception. > We introduce SpaceAct, an interactive benchmark evaluating 3D spatiotemporal reasoning via closed-loop physical action and active perception - [Supporting source](https://arxiv.org/abs/2610.38405) ## Source-stated The benchmark contains 108 verified instances across 36 task families in six categories, without privileged coordinates. > it has 108 verified instances across 36 task families in six categories, without privileged coordinates - [Supporting source](https://arxiv.org/abs/2610.38405) ## Source-stated On the 44 hardest instances, scores across 10 frontier MLLMs ranged from 4.8 to 79.1. > Across 10 frontier MLLMs on the 44 hardest instances, scores span 4.8-79.1 - [Supporting source](https://arxiv.org/abs/2610.38405) ## Source-stated For the top three models, adding a reference plan closed 44% of the gap but left collisions at 5.2 per episode; adding exact 3D state closed 90% and reduced collisions to 0.5. > for the top three, a reference plan (+plan) closes 44% of the gap while leaving collisions at 5.2 per episode, whereas adding exact 3D state (+state) closes 90% and cuts collisions to 0.5 - [Supporting source](https://arxiv.org/abs/2610.38405) ## Source-stated The authors identify 3D visual grounding and action refinement as bottlenecks, even when a valid plan is available. > showing that even with a valid plan, MLLMs remain bottlenecked by 3D visual grounding and action refinement - [Supporting source](https://arxiv.org/abs/2610.38405) ## Source and contributor context Shi, Chenglong, Yuan, Zekai, Song, Yue, Pu, Jon, Wu, Will · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## Multimodal model A model that works with more than one kind of input or output, such as text, images and audio. - [Definition reference](https://developers.google.com/machine-learning/glossary#multimodal-model) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 6a040580-cb78-4ab7-8530-ca370acdfdb1. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-13T04:17:21.982Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 65d82fb5517c5e8660271fd8f432a4a59afe47d24a669578b1f48586d23e3a71. Source text hash: 74dd5d5028e767ae2193c744e2c361c88a0740c8a85173f1bc47c9696116f98f. ## Revision and source-check history ### Version 1 — 2026-11-13T04:17:21.982Z The authors introduce SpaceAct, an interactive benchmark for testing whether multimodal large language models (MLLMs) can reason about 3D space through physical action, rather than only describe images or plan symbolically. Its tasks use interlocking bricks and mechanical puzzles, with models able to rotate a camera, inspect rulers, issue 6-DoF poses, and adjust actions using feedback. The benchmark is designed to test whether models can handle occlusion, collision, statics, and timing without privileged coordinates. Across 10 frontier MLLMs on the 44 hardest instances, scores ranged from 4.8 to 79.1. The authors report that adding a reference plan helped some models, but collisions remained; providing exact 3D state improved scores and cut collisions in the reported comparisons. They identify collision or observation loops as 46% of failures. Their central argument is that a valid plan is not enough: visual grounding and action refinement remain bottlenecks in these tasks. SpaceAct offers a testbed for evaluating active multi-view exploration and explicit 3D grounding, but the abstract does not establish how performance transfers beyond its benchmark. How well would performance on SpaceAct predict MLLM behavior in physical settings beyond these puzzles? ## Original source arXiv cs.AI · Article Original source date: Oct 28, 2026 (2026-10-28). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-13T04:17:21.982Z. First observed: 2026-10-30T06:40:25.159Z. Retrieved: 2026-10-31T08:16:51.562Z. - [SpaceAct: Benchmarking 3D Spatial and Temporal Reasoning via Closed-Loop Action and Active Perception](https://arxiv.org/abs/2610.38405) ## Limits The supplied evidence is the arXiv abstract, so it does not provide details needed to assess the benchmark design or reported comparisons beyond the summary figures. The abstract reports benchmark results but does not establish whether performance predicts behavior in physical settings beyond the benchmark's interlocking-brick and mechanical-puzzle tasks. ## Open question How well would performance on SpaceAct predict MLLM behavior in physical settings beyond these puzzles? ## Source-stated The authors present SpaceAct as an interactive benchmark for evaluating 3D spatiotemporal reasoning through physical action and active perception. > We introduce SpaceAct, an interactive benchmark evaluating 3D spatiotemporal reasoning via closed-loop physical action and active perception - [Supporting source](https://arxiv.org/abs/2610.38405) ## Source-stated The benchmark contains 108 verified instances across 36 task families in six categories, without privileged coordinates. > it has 108 verified instances across 36 task families in six categories, without privileged coordinates - [Supporting source](https://arxiv.org/abs/2610.38405) ## Source-stated On the 44 hardest instances, scores across 10 frontier MLLMs ranged from 4.8 to 79.1. > Across 10 frontier MLLMs on the 44 hardest instances, scores span 4.8-79.1 - [Supporting source](https://arxiv.org/abs/2610.38405) ## Source-stated For the top three models, adding a reference plan closed 44% of the gap but left collisions at 5.2 per episode; adding exact 3D state closed 90% and reduced collisions to 0.5. > for the top three, a reference plan (+plan) closes 44% of the gap while leaving collisions at 5.2 per episode, whereas adding exact 3D state (+state) closes 90% and cuts collisions to 0.5 - [Supporting source](https://arxiv.org/abs/2610.38405) ## Source-stated The authors identify 3D visual grounding and action refinement as bottlenecks, even when a valid plan is available. > showing that even with a valid plan, MLLMs remain bottlenecked by 3D visual grounding and action refinement - [Supporting source](https://arxiv.org/abs/2610.38405) ## Source and contributor context Shi, Chenglong, Yuan, Zekai, Song, Yue, Pu, Jon, Wu, Will · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## Multimodal model A model that works with more than one kind of input or output, such as text, images and audio. - [Definition reference](https://developers.google.com/machine-learning/glossary#multimodal-model) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 6a040580-cb78-4ab7-8530-ca370acdfdb1. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-13T04:17:21.982Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 65d82fb5517c5e8660271fd8f432a4a59afe47d24a669578b1f48586d23e3a71. Source text hash: 74dd5d5028e767ae2193c744e2c361c88a0740c8a85173f1bc47c9696116f98f. ### passed — 2026-11-13T04:17:21.982Z — anthropic/claude-opus-5.5 Publication: 6a040580-cb78-4ab7-8530-ca370acdfdb1 attribution — pass: The reading attributes SpaceAct, the results and the bottleneck conclusion to the paper's authors. Its framing as an arXiv abstract matches the evidence, and the authors' interpretation is not presented as established fact. > Authors:Chenglong Shi, Zekai Yuan, Yue Song, Jon Pu, Will Wu summary — pass: The reading leads with what the authors introduce, explains MLLMs and how the benchmark differs from describing images or planning symbolically, and conveys the central argument that a valid plan is not enough. Terms such as 6-DoF, privileged coordinates and statics are kept exactly, consistent with tooltip handling. The question follows from the evidence and is not leading. Minor nit: the closing testbed sentence could add 'the authors say', but the meaning is faithful. > showing that even with a valid plan, MLLMs remain bottlenecked by 3D visual grounding and action refinement date — pass: The originalPublished date of 2026-10-28 matches the arXiv v1 submission date. The later feed date of Oct 31 is correctly not used. > \[Submitted on 28 Oct 2026\] support — pass: Each claim excerpt is verbatim from the abstract. The summary's statements about plan-only gains with remaining collisions, state-based gains, and the 46% failure share match the source. The limitations correctly note that transfer beyond the benchmark is unestablished, and no implied benefits or costs are invented. > and 46% of failures are collision or observation loops media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/8de72948dc437d80a542533d80feef28470a28c438470c99e13001aa84bc64b4/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/spaceact-benchmarking-3d-spatial-and-temporal-reasoning-via-8de72948.json)