Barhdadi, Mohamed Rayan, Wasim, Syed Talal, Gall, Juergen, Serpedin, Erchin, Kurban, Hasan · Nov 7, 2026
arXiv q-bio.NC: Perception, Not Reasoning, Limits Visual Theory of Mind
Markdown · JSON · Original source
The authors of the paper argue that vision-language models struggle with visual theory of mind mainly because they fail to perceive who was present and where objects were, not because they cannot reason about beliefs. They compare the same reasoning task across five conditions that vary perceptual fidelity, from visual-only to text-only, and introduce the Perceptual Contribution Ratio to measure how much each condition closes the visual-to-text performance gap. Across abstract 2D panels and naturalistic 3D game scenes, ground-truth scaffolds substantially narrowed that gap; on the abstract panels and MuMA-ToM, they matched or exceeded text-only reasoning. Automatic-parser scaffolds, including those built by frontier VLMs, failed. The authors identify reality bias—reporting an object’s actual location rather than the location an absent agent believes it to be—as the most common visual-only false-belief error. They argue that structured visual scene-state extraction is the dominant bottleneck under their evaluated protocol. The result challenges an account that treats poor visual performance chiefly as a reasoning problem, while leaving open how broadly the finding applies beyond the tested models and scenes.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
q-bio.NC · Article
Source details
Original source date: Nov 7, 2026 (2026-11-07).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-12T12:34:32.749Z. First observed: 2026-11-10T08:34:13.297Z. Retrieved: 2026-11-12T12:33:33.376Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The reported conclusion is explicitly limited to the evaluated protocol; the evidence describes two scene types and seven VLMs, so broader generalization remains unestablished. The evidence is the abstract, not the full paper; it does not provide detailed methods, results, or uncertainty estimates.
Open question
Would the authors’ finding that structured visual scene-state extraction is the dominant bottleneck hold beyond the evaluated models and scene types?
Source-stated
The authors argue that visual theory-of-mind performance is limited chiefly by perception rather than belief reasoning.
We argue the dominant bottleneck is perceptual: not what the model cannot reason about, but what it cannot see.
Source-stated
The paper tests a protocol that varies perceptual fidelity while holding the reasoning task constant.
a five-condition dissociation protocol that holds the reasoning task constant while varying perceptual fidelity from visual only to text only
Source-stated
Ground-truth scaffolds substantially narrow the visual-to-text gap, while automatic-parser scaffolds fail.
Ground-truth scaffolds substantially close the gap in every setting, and on the abstract panels and MuMA-ToM they match or exceed the text-only reasoning. Whereas scaffolds built by automatic parsers, including frontier VLMs, fail.
Source-stated
The authors identify reality bias as the most common visual-only false-belief failure.
The most common error in the visual-only condition is \emph{reality bias}, which accounts for 60\% of false-belief failures.
Source-stated
Under the evaluated protocol, the authors identify structured visual scene-state extraction as the dominant bottleneck.
Under the evaluated protocol, structured visual scene-state extraction, rather than belief inference, is the dominant bottleneck for visual ToM.
Source-stated
The evaluation covers seven VLMs and roughly 159,000 API calls, across abstract panels and naturalistic game scenes.
We test this on two kinds of scenes: abstract 2D panels in the Heider--Simmel tradition, and naturalistic 3D game scenes from MuMA-ToM and MindPower. The evaluation covers seven VLMs and roughly 159,000 API calls.
Source and contributor context
Barhdadi, Mohamed Rayan, Wasim, Syed Talal, Gall, Juergen, Serpedin, Erchin, Kurban, Hasan · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 15b3ccb0-96f6-4d77-8999-9f25569a2d81.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-12T12:34:32.749Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: d271088841173a80d3c626d6fdcc43f3136093b16a8e61a8111d17076702be3d. Source text hash: 1834f8247a55d8355469f9a6077500e2d98524f48961838caa71da7cc51b66a0.
Revision and source-check history
Version 1 · 2026-11-12T12:34:32.749Z
arXiv q-bio.NC: Perception, Not Reasoning, Limits Visual Theory of Mind
The authors of the paper argue that vision-language models struggle with visual theory of mind mainly because they fail to perceive who was present and where objects were, not because they cannot reason about beliefs. They compare the same reasoning task across five conditions that vary perceptual fidelity, from visual-only to text-only, and introduce the Perceptual Contribution Ratio to measure how much each condition closes the visual-to-text performance gap. Across abstract 2D panels and naturalistic 3D game scenes, ground-truth scaffolds substantially narrowed that gap; on the abstract panels and MuMA-ToM, they matched or exceeded text-only reasoning. Automatic-parser scaffolds, including those built by frontier VLMs, failed. The authors identify reality bias—reporting an object’s actual location rather than the location an absent agent believes it to be—as the most common visual-only false-belief error. They argue that structured visual scene-state extraction is the dominant bottleneck under their evaluated protocol. The result challenges an account that treats poor visual performance chiefly as a reasoning problem, while leaving open how broadly the finding applies beyond the tested models and scenes.
Original source
q-bio.NC · Article
Source details
Original source date: Nov 7, 2026 (2026-11-07).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-12T12:34:32.749Z. First observed: 2026-11-10T08:34:13.297Z. Retrieved: 2026-11-12T12:33:33.376Z.
Limits
The reported conclusion is explicitly limited to the evaluated protocol; the evidence describes two scene types and seven VLMs, so broader generalization remains unestablished. The evidence is the abstract, not the full paper; it does not provide detailed methods, results, or uncertainty estimates.
Open question
Would the authors’ finding that structured visual scene-state extraction is the dominant bottleneck hold beyond the evaluated models and scene types?
Source-stated
The authors argue that visual theory-of-mind performance is limited chiefly by perception rather than belief reasoning.
We argue the dominant bottleneck is perceptual: not what the model cannot reason about, but what it cannot see.
Source-stated
The paper tests a protocol that varies perceptual fidelity while holding the reasoning task constant.
a five-condition dissociation protocol that holds the reasoning task constant while varying perceptual fidelity from visual only to text only
Source-stated
Ground-truth scaffolds substantially narrow the visual-to-text gap, while automatic-parser scaffolds fail.
Ground-truth scaffolds substantially close the gap in every setting, and on the abstract panels and MuMA-ToM they match or exceed the text-only reasoning. Whereas scaffolds built by automatic parsers, including frontier VLMs, fail.
Source-stated
The authors identify reality bias as the most common visual-only false-belief failure.
The most common error in the visual-only condition is \emph{reality bias}, which accounts for 60\% of false-belief failures.
Source-stated
Under the evaluated protocol, the authors identify structured visual scene-state extraction as the dominant bottleneck.
Under the evaluated protocol, structured visual scene-state extraction, rather than belief inference, is the dominant bottleneck for visual ToM.
Source-stated
The evaluation covers seven VLMs and roughly 159,000 API calls, across abstract panels and naturalistic game scenes.
We test this on two kinds of scenes: abstract 2D panels in the Heider--Simmel tradition, and naturalistic 3D game scenes from MuMA-ToM and MindPower. The evaluation covers seven VLMs and roughly 159,000 API calls.
Source and contributor context
Barhdadi, Mohamed Rayan, Wasim, Syed Talal, Gall, Juergen, Serpedin, Erchin, Kurban, Hasan · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 15b3ccb0-96f6-4d77-8999-9f25569a2d81.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-12T12:34:32.749Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: d271088841173a80d3c626d6fdcc43f3136093b16a8e61a8111d17076702be3d. Source text hash: 1834f8247a55d8355469f9a6077500e2d98524f48961838caa71da7cc51b66a0.
uncertain · 2026-11-12T12:34:32.749Z · anthropic/claude-opus-5.5
Publication: 15b3ccb0-96f6-4d77-8999-9f25569a2d81
attribution — pass: The argument is attributed to the paper's authors throughout, and the conclusion stays tied to their evaluated protocol. The title prefix matches the supplied publisher field. The paper's primary listing is cs.CV and it is cross-listed in q-bio.NC, so the prefix is not a misattribution.
Under the evaluated protocol, structured visual scene-state extraction, rather than belief inference, is the dominant bottleneck for visual ToM.
summary — uncertain: The reading omits the paper's motivating finding: VLMs perform near-human on text false-belief probes but drop more than twenty percentage points when the same problem is shown visually. Without it, the visual-to-text gap and its stakes are hard to follow. It also softens the challenge to a stated consensus that prescribes new belief-reasoning modules. Central terms such as theory of mind and ground-truth scaffolds are left unexplained. Smallest repair: add one sentence on the text-versus-visual drop and the consensus fix, and add tooltips for theory of mind, false-belief and ground-truth scaffold.
Vision-language models pass text-based false-belief probes at near-human levels but drop by over twenty percentage points on abstract panels the moment the same problem is conveyed visually.
date — pass: originalPublished is 2026-11-07, which matches the v1 submission date. It is not the later feed date of 2026-11-10.
[Submitted on 7 Nov 2026]
support — pass: Every claim matches a verbatim source excerpt. The limitations accurately note the protocol scope and that only the abstract was available. The phrase about failing to perceive where objects were slightly extends the source, which speaks of tracking which agents were present. It remains a reasonable gloss of reality bias, and the open question follows from the stated scope.
This points to a perceptual front-end that fails to track which agents were present at each moment.
media — pass: The selected media is the verified arXiv abstract page, the direct original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv