Gu, Tianjun, Gong, Jingyu, Zhang, Zhizhong, Xie, Yuan, Ma, Lizhuang, Tan, Xin, Vasilakos, Athanasios V. · Jan 4, 2026
arXiv cs.AI: Vision-language models lag human performance on physical dynamics and intent reasoning
Markdown · JSON · Original source
The paper’s authors argue that vision-language models can perform strongly on controlled yet struggle to reason jointly about physical interactions, reference frames and human intentions in open-world environments. They define Teleo-Spatial Intelligence (TSI) as integrating physical dynamics and action goals to reason about actions and their spatial consequences, and introduce EscherVerse to evaluate it. The resource includes real-world videos, an 8,000-example benchmark and an instruction-tuning set. Across 27 models, the strongest proprietary model scored 57.26% overall accuracy; first-pass human performance ranged from 84.81% to 95.14%, with a mean of 90.62%. The authors report that on real-world, intent-aware data narrows, but does not close, this gap. Their central point is that recognizing visual patterns is not the same as understanding how physical change relates to what a person is trying to do. EscherVerse is presented as a diagnostic testbed for that purpose-aware spatial reasoning. The abstract reports these comparisons, but does not specify which aspects of the task account for the remaining difference.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Jan 4, 2026 (2026-01-04).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-16T07:03:14.720Z. First observed: 2026-11-16T05:06:37.239Z. Retrieved: 2026-11-16T07:02:16.826Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The evidence provided is the paper’s abstract and publisher page; it does not include the full paper’s methods or detailed comparison conditions. The abstract reports that narrows the gap but does not specify which aspects of the tasks account for the remaining difference.
Open question
Which aspects of physical dynamics, reference frames or inferred human intentions account for the remaining gap after ?
Source-stated
The authors define Teleo-Spatial Intelligence as integrating physical dynamics and action goals to reason about actions and their spatial consequences.
integrating physical dynamics and action goals to reason about actions and their spatial consequences
Source-stated
EscherVerse includes an 8,000-example benchmark and a 35,963-example instruction-tuning set, built from 11,328 real-world videos.
built from 11,328 real-world videos, including an 8,000-example benchmark and a 35,963-example instruction-tuning set
Source-stated
Across 27 vision-language models, the strongest proprietary model achieved 57.26% overall accuracy; first-pass human performance ranged from 84.81% to 95.14%, with a mean of 90.62%.
the strongest proprietary model achieves 57.26% overall accuracy, far below first-pass human performance, which ranges from 84.81% to 95.14% with a mean of 90.62%
Source-stated
The authors report that fine-tuning open-weight models on real-world, intent-aware data narrows but does not close the reported gap.
Fine-tuning on real-world, intent-aware data narrows this gap for open-weight models, but does not close it.
Source and contributor context
Gu, Tianjun, Gong, Jingyu, Zhang, Zhizhong, Xie, Yuan, Ma, Lizhuang, Tan, Xin, Vasilakos, Athanasios V. · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Fine-tuning
Further training of an existing model to adapt it to a particular task or use.
Open weights
A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: c2681b91-4d0e-4731-8034-995ef9617d72.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-16T07:03:14.720Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: fbe5364046e7386b4577297bec7ca7b06fad65dd85b1db4d7662428174944a98. Source text hash: d8cf26837c5d2ccb9269e3313f9763c123c9f363b53980fd80544b174f25d935.
Revision and source-check history
Version 1 · 2026-11-16T07:03:14.720Z
arXiv cs.AI: Vision-language models lag human performance on physical dynamics and intent reasoning
The paper’s authors argue that vision-language models can perform strongly on controlled yet struggle to reason jointly about physical interactions, reference frames and human intentions in open-world environments. They define Teleo-Spatial Intelligence (TSI) as integrating physical dynamics and action goals to reason about actions and their spatial consequences, and introduce EscherVerse to evaluate it. The resource includes real-world videos, an 8,000-example benchmark and an instruction-tuning set. Across 27 models, the strongest proprietary model scored 57.26% overall accuracy; first-pass human performance ranged from 84.81% to 95.14%, with a mean of 90.62%. The authors report that on real-world, intent-aware data narrows, but does not close, this gap. Their central point is that recognizing visual patterns is not the same as understanding how physical change relates to what a person is trying to do. EscherVerse is presented as a diagnostic testbed for that purpose-aware spatial reasoning. The abstract reports these comparisons, but does not specify which aspects of the task account for the remaining difference.
Original source
cs.AI · Article
Source details
Original source date: Jan 4, 2026 (2026-01-04).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-16T07:03:14.720Z. First observed: 2026-11-16T05:06:37.239Z. Retrieved: 2026-11-16T07:02:16.826Z.
Limits
The evidence provided is the paper’s abstract and publisher page; it does not include the full paper’s methods or detailed comparison conditions. The abstract reports that narrows the gap but does not specify which aspects of the tasks account for the remaining difference.
Open question
Which aspects of physical dynamics, reference frames or inferred human intentions account for the remaining gap after ?
Source-stated
The authors define Teleo-Spatial Intelligence as integrating physical dynamics and action goals to reason about actions and their spatial consequences.
integrating physical dynamics and action goals to reason about actions and their spatial consequences
Source-stated
EscherVerse includes an 8,000-example benchmark and a 35,963-example instruction-tuning set, built from 11,328 real-world videos.
built from 11,328 real-world videos, including an 8,000-example benchmark and a 35,963-example instruction-tuning set
Source-stated
Across 27 vision-language models, the strongest proprietary model achieved 57.26% overall accuracy; first-pass human performance ranged from 84.81% to 95.14%, with a mean of 90.62%.
the strongest proprietary model achieves 57.26% overall accuracy, far below first-pass human performance, which ranges from 84.81% to 95.14% with a mean of 90.62%
Source-stated
The authors report that fine-tuning open-weight models on real-world, intent-aware data narrows but does not close the reported gap.
Fine-tuning on real-world, intent-aware data narrows this gap for open-weight models, but does not close it.
Source and contributor context
Gu, Tianjun, Gong, Jingyu, Zhang, Zhizhong, Xie, Yuan, Ma, Lizhuang, Tan, Xin, Vasilakos, Athanasios V. · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Fine-tuning
Further training of an existing model to adapt it to a particular task or use.
Open weights
A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: c2681b91-4d0e-4731-8034-995ef9617d72.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-16T07:03:14.720Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: fbe5364046e7386b4577297bec7ca7b06fad65dd85b1db4d7662428174944a98. Source text hash: d8cf26837c5d2ccb9269e3313f9763c123c9f363b53980fd80544b174f25d935.
passed · 2026-11-16T07:03:14.720Z · anthropic/claude-opus-5.5
Publication: c2681b91-4d0e-4731-8034-995ef9617d72
attribution — pass: The reading attributes the argument, the TSI definition, the EscherVerse resource and the results to the paper's authors. arXiv is presented only as the publisher. Reported findings stay framed as the authors' claims, not as Escape Meridian conclusions.
Authors:Tianjun Gu, Jingyu Gong, Zhizhong Zhang, Yuan Xie, Lizhuang Ma, Xin Tan, Athanasios V. Vasilakos
summary — pass: The summary leads with the authors' argument and explains TSI in plain terms. It gives the key benchmark comparison and the fine-tuning result, and it preserves the 'narrows but does not close' condition. The pattern-recognition versus understanding point closely tracks the abstract. The question names a real unresolved issue the abstract leaves open. Terms such as vision-language model and open-weight are left as technical terms for tooltips. Optional additions would be that the human baseline came from 11 annotators and that the abstract frames this in terms of embodied AI.
highlights a critical gap between pattern recognition and human-level understanding in embodied AI
date — pass: The displayed date of 2026-01-04 matches the v1 submission date. It is not the date of the later v3 revision (13 Nov 2026) or the feed entry date.
[Submitted on 4 Jan 2026 (v1), last revised 13 Nov 2026 (this version, v3)]
support — pass: Every claim's excerpt matches the source exactly, and the numbers are reproduced correctly. The reading does not turn the benchmark scores into claims about real-world reliability. The limitations accurately note that only the abstract and page are available. The question follows from the source and implies no unsupported benefits or costs.
Fine-tuning on real-world, intent-aware data narrows this gap for open-weight models, but does not close it.
media — pass: The selected media is the verified direct link to the original arXiv abstract page. It matches the source URL and is the most direct way for readers to reach the paper.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceFurther training of an existing model to adapt it to a particular task or use.
Definition referenceA model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivFurther training of an existing model to adapt it to a particular task or use.
Definition referenceFurther training of an existing model to adapt it to a particular task or use.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceFurther training of an existing model to adapt it to a particular task or use.
Definition referenceA model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivFurther training of an existing model to adapt it to a particular task or use.
Definition referenceFurther training of an existing model to adapt it to a particular task or use.
Definition reference