Qiuyi, Zhang, Xu, Zhen, Gu, Jing, Bang, Seojin, Das, Arnav Mohanty, Hu, Yaning, Kumar, Gautam, Lichtarge, Jared, Xue, Le, Catoto, Bettina, Datta, Akul, Luo, Daniel, Khan, Farha, Valentine, Marcella, Yang, Yinfei, Dai, Andrew M. · Oct 26, 2026
arXiv cs.AI paper DecoyCount finds a gap between cued and unaided visual counting
Markdown · JSON · Original source
In the DecoyCount paper, Qiuyi (Richard) Zhang and coauthors argue that visual counting errors can reflect a mismatch between a model’s reasoning and the visual task, not just difficulty counting. Their uses real-world images with designed decoys and compares answers to a neutral question with answers when a cue points the model toward the intended counting process. The authors call the difference in accuracy the reasoning-alignment gap: models can count when cued but fail to select that process on their own. The authors report that scored below 50% accuracy on the benchmark, with gaps of 12.1–22.1 percentage points. They also report that greater model capability, larger thinking budgets and processes often came with larger gaps. Adding scenario context without an explicit hint did not improve aggregate performance. The paper’s argument is that stronger general reasoning alone does not ensure visual reasoning aligned with human expectations. This distinction matters when interpreting a correct answer under prompting: it may show that a model can carry out a counting process, without showing that it will choose that process unaided. These results concern the paper’s benchmark; how far they extend to other visual tasks is unresolved.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 26, 2026 (2026-10-26).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-12T14:31:55.723Z. First observed: 2026-10-28T11:58:40.196Z. Retrieved: 2026-11-12T14:03:04.542Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The reported accuracy and reasoning-alignment gaps are results on the DecoyCount ; the supplied abstract does not establish how they generalize to other visual tasks. The supplied evidence is the paper’s publisher-page abstract and does not provide further methodological detail or results beyond its summary.
Open question
Would the reported reasoning-alignment gaps persist on visual tasks beyond the counting problems in DecoyCount?
Source-stated
The authors introduce DecoyCount, a benchmark using real-world images with designed decoys and paired prompts to evaluate visual counting.
We introduce DecoyCount, a benchmark built from 1,586 real-world images with designed decoys across seven mechanisms
Source-stated
The authors define the reasoning-alignment gap as the accuracy difference between cue-guided and unaided counting.
we term this difference in accuracy the reasoning-alignment gap
Source-stated
The authors report frontier models scored below 50% accuracy, with reasoning-alignment gaps from 12.1 to 22.1 percentage points on their benchmark.
frontier models remain below 50% accuracy on our benchmark, while alignment gaps range from 12.1-22.1 percentage points
Source-stated
The authors report that greater model capability, larger thinking budgets and agentic processes often induce larger gaps.
more capable models, greater thinking budgets, and agentic processes often induce larger gaps
Source-stated
The authors report that added scenario context without explicit hinting did not improve aggregate performance.
this added context does not improve aggregate performance
Source-stated
The authors conclude that stronger general reasoning does not by itself generalize to visual reasoning aligned with human expectations.
stronger general reasoning does not by itself generalize to aligned visual reasoning
Source and contributor context
Qiuyi, Zhang, Xu, Zhen, Gu, Jing, Bang, Seojin, Das, Arnav Mohanty, Hu, Yaning, Kumar, Gautam, Lichtarge, Jared, Xue, Le, Catoto, Bettina, Datta, Akul, Luo, Daniel, Khan, Farha, Valentine, Marcella, Yang, Yinfei, Dai, Andrew M. · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Frontier model
One of the most capable AI models available at a given time; the set shifts as models improve.
Agentic AI
AI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 6b144c26-5746-49a1-8cd7-f718149c2a1f.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-12T14:31:55.723Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 4ed8fdda52bb80c188646d543a532fdcf6a3cd4bce0b2bc0f03f0ccb52d81d8e. Source text hash: 0e87d8e209ee57c5f19c200e5797bc72b3b5caa02628dc6e5f4c42f203b7aee9.
Revision and source-check history
Version 1 · 2026-11-12T14:31:55.723Z
arXiv cs.AI paper DecoyCount finds a gap between cued and unaided visual counting
In the DecoyCount paper, Qiuyi (Richard) Zhang and coauthors argue that visual counting errors can reflect a mismatch between a model’s reasoning and the visual task, not just difficulty counting. Their uses real-world images with designed decoys and compares answers to a neutral question with answers when a cue points the model toward the intended counting process. The authors call the difference in accuracy the reasoning-alignment gap: models can count when cued but fail to select that process on their own. The authors report that scored below 50% accuracy on the benchmark, with gaps of 12.1–22.1 percentage points. They also report that greater model capability, larger thinking budgets and processes often came with larger gaps. Adding scenario context without an explicit hint did not improve aggregate performance. The paper’s argument is that stronger general reasoning alone does not ensure visual reasoning aligned with human expectations. This distinction matters when interpreting a correct answer under prompting: it may show that a model can carry out a counting process, without showing that it will choose that process unaided. These results concern the paper’s benchmark; how far they extend to other visual tasks is unresolved.
Original source
cs.AI · Article
Source details
Original source date: Oct 26, 2026 (2026-10-26).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-12T14:31:55.723Z. First observed: 2026-10-28T11:58:40.196Z. Retrieved: 2026-11-12T14:03:04.542Z.
Limits
The reported accuracy and reasoning-alignment gaps are results on the DecoyCount ; the supplied abstract does not establish how they generalize to other visual tasks. The supplied evidence is the paper’s publisher-page abstract and does not provide further methodological detail or results beyond its summary.
Open question
Would the reported reasoning-alignment gaps persist on visual tasks beyond the counting problems in DecoyCount?
Source-stated
The authors introduce DecoyCount, a benchmark using real-world images with designed decoys and paired prompts to evaluate visual counting.
We introduce DecoyCount, a benchmark built from 1,586 real-world images with designed decoys across seven mechanisms
Source-stated
The authors define the reasoning-alignment gap as the accuracy difference between cue-guided and unaided counting.
we term this difference in accuracy the reasoning-alignment gap
Source-stated
The authors report frontier models scored below 50% accuracy, with reasoning-alignment gaps from 12.1 to 22.1 percentage points on their benchmark.
frontier models remain below 50% accuracy on our benchmark, while alignment gaps range from 12.1-22.1 percentage points
Source-stated
The authors report that greater model capability, larger thinking budgets and agentic processes often induce larger gaps.
more capable models, greater thinking budgets, and agentic processes often induce larger gaps
Source-stated
The authors report that added scenario context without explicit hinting did not improve aggregate performance.
this added context does not improve aggregate performance
Source-stated
The authors conclude that stronger general reasoning does not by itself generalize to visual reasoning aligned with human expectations.
stronger general reasoning does not by itself generalize to aligned visual reasoning
Source and contributor context
Qiuyi, Zhang, Xu, Zhen, Gu, Jing, Bang, Seojin, Das, Arnav Mohanty, Hu, Yaning, Kumar, Gautam, Lichtarge, Jared, Xue, Le, Catoto, Bettina, Datta, Akul, Luo, Daniel, Khan, Farha, Valentine, Marcella, Yang, Yinfei, Dai, Andrew M. · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Frontier model
One of the most capable AI models available at a given time; the set shifts as models improve.
Agentic AI
AI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 6b144c26-5746-49a1-8cd7-f718149c2a1f.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-12T14:31:55.723Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 4ed8fdda52bb80c188646d543a532fdcf6a3cd4bce0b2bc0f03f0ccb52d81d8e. Source text hash: 0e87d8e209ee57c5f19c200e5797bc72b3b5caa02628dc6e5f4c42f203b7aee9.
passed · 2026-11-12T14:31:55.723Z · anthropic/claude-opus-5.5
Publication: 6b144c26-5746-49a1-8cd7-f718149c2a1f
attribution — pass: The draft attributes the paper to Qiuyi (Richard) Zhang and coauthors and identifies it as an arXiv cs.AI paper, matching the author list and listing. Findings and conclusions remain framed as the authors' reports rather than established fact.
Authors:Qiuyi (Richard)Zhang, Zhen Xu, Jing Gu
summary — pass: The summary accurately conveys the benchmark design: real-world images with designed decoys, neutral versus cue-guided prompts, and a scenario condition. It also defines the reasoning-alignment gap and reports the headline results and conclusion. The title and question follow from the evidence. Terms such as thinking budgets and agentic processes are kept exactly as technical terms and suit tooltips. The central idea is readable: models can count when cued but fail to choose that process unaided.
models can execute the intended counting process when cue-guided but fail to select it unaided
date — pass: originalPublished is 2026-10-26, which matches the v1 submission date. The 5 Nov 2026 date is a later revision, and the 9 Nov 2026 date is the feed entry, so neither should be displayed as the original publication date.
Submitted on 26 Oct 2026 (v1), last revised 5 Nov 2026
support — pass: Every claim is supported verbatim by the abstract: the sub-50% accuracy, the 12.1–22.1 point gaps, larger gaps with more capability and thinking, no gain from added context, and the conclusion. The interpretive sentence about correct answers under prompting closely tracks the source's cue-guided versus unaided finding. The draft scopes the results to the benchmark and leaves generalization as an open question, without unsupported benefits or costs.
this added context does not improve aggregate performance
media — pass: The selected media is the verified direct link to the arXiv abstract page, which is the original source. No other source media was found.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceOne of the most capable AI models available at a given time; the set shifts as models improve.
Definition referenceAI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceOne of the most capable AI models available at a given time; the set shifts as models improve.
Definition referenceAI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference