# arXiv cs.AI paper DecoyCount finds a gap between cued and unaided visual counting [Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-paper-decoycount-finds-a-gap-between-cued-and-9ae65e60) In the DecoyCount paper, Qiuyi (Richard) Zhang and coauthors argue that visual counting errors can reflect a mismatch between a model’s reasoning and the visual task, not just difficulty counting. Their benchmark uses real-world images with designed decoys and compares answers to a neutral question with answers when a cue points the model toward the intended counting process. The authors call the difference in accuracy the reasoning-alignment gap: models can count when cued but fail to select that process on their own. The authors report that frontier models scored below 50% accuracy on the benchmark, with gaps of 12.1–22.1 percentage points. They also report that greater model capability, larger thinking budgets and agentic processes often came with larger gaps. Adding scenario context without an explicit hint did not improve aggregate performance. The paper’s argument is that stronger general reasoning alone does not ensure visual reasoning aligned with human expectations. This distinction matters when interpreting a correct answer under prompting: it may show that a model can carry out a counting process, without showing that it will choose that process unaided. These results concern the paper’s benchmark; how far they extend to other visual tasks is unresolved. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Oct 26, 2026 (2026-10-26). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-12T14:31:55.723Z. First observed: 2026-10-28T11:58:40.196Z. Retrieved: 2026-11-12T14:03:04.542Z. - [DecoyCount: Evaluating Visual-Reasoning Misalignment in Counting](https://arxiv.org/abs/2610.34855) ## Limits The reported accuracy and reasoning-alignment gaps are results on the DecoyCount benchmark; the supplied abstract does not establish how they generalize to other visual tasks. The supplied evidence is the paper’s publisher-page abstract and does not provide further methodological detail or results beyond its summary. ## Open question Would the reported reasoning-alignment gaps persist on visual tasks beyond the counting problems in DecoyCount? ## Source-stated The authors introduce DecoyCount, a benchmark using real-world images with designed decoys and paired prompts to evaluate visual counting. > We introduce DecoyCount, a benchmark built from 1,586 real-world images with designed decoys across seven mechanisms - [Supporting source](https://arxiv.org/abs/2610.34855) ## Source-stated The authors define the reasoning-alignment gap as the accuracy difference between cue-guided and unaided counting. > we term this difference in accuracy the reasoning-alignment gap - [Supporting source](https://arxiv.org/abs/2610.34855) ## Source-stated The authors report frontier models scored below 50% accuracy, with reasoning-alignment gaps from 12.1 to 22.1 percentage points on their benchmark. > frontier models remain below 50% accuracy on our benchmark, while alignment gaps range from 12.1-22.1 percentage points - [Supporting source](https://arxiv.org/abs/2610.34855) ## Source-stated The authors report that greater model capability, larger thinking budgets and agentic processes often induce larger gaps. > more capable models, greater thinking budgets, and agentic processes often induce larger gaps - [Supporting source](https://arxiv.org/abs/2610.34855) ## Source-stated The authors report that added scenario context without explicit hinting did not improve aggregate performance. > this added context does not improve aggregate performance - [Supporting source](https://arxiv.org/abs/2610.34855) ## Source-stated The authors conclude that stronger general reasoning does not by itself generalize to visual reasoning aligned with human expectations. > stronger general reasoning does not by itself generalize to aligned visual reasoning - [Supporting source](https://arxiv.org/abs/2610.34855) ## Source and contributor context Qiuyi, Zhang, Xu, Zhen, Gu, Jing, Bang, Seojin, Das, Arnav Mohanty, Hu, Yaning, Kumar, Gautam, Lichtarge, Jared, Xue, Le, Catoto, Bettina, Datta, Akul, Luo, Daniel, Khan, Farha, Valentine, Marcella, Yang, Yinfei, Dai, Andrew M. · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## Frontier model One of the most capable AI models available at a given time; the set shifts as models improve. - [Definition reference](https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper) ## Agentic AI AI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer. - [Definition reference](https://developers.google.com/machine-learning/glossary#agentic) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 6b144c26-5746-49a1-8cd7-f718149c2a1f. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-12T14:31:55.723Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 4ed8fdda52bb80c188646d543a532fdcf6a3cd4bce0b2bc0f03f0ccb52d81d8e. Source text hash: 0e87d8e209ee57c5f19c200e5797bc72b3b5caa02628dc6e5f4c42f203b7aee9. ## Revision and source-check history ### Version 1 — 2026-11-12T14:31:55.723Z In the DecoyCount paper, Qiuyi (Richard) Zhang and coauthors argue that visual counting errors can reflect a mismatch between a model’s reasoning and the visual task, not just difficulty counting. Their benchmark uses real-world images with designed decoys and compares answers to a neutral question with answers when a cue points the model toward the intended counting process. The authors call the difference in accuracy the reasoning-alignment gap: models can count when cued but fail to select that process on their own. The authors report that frontier models scored below 50% accuracy on the benchmark, with gaps of 12.1–22.1 percentage points. They also report that greater model capability, larger thinking budgets and agentic processes often came with larger gaps. Adding scenario context without an explicit hint did not improve aggregate performance. The paper’s argument is that stronger general reasoning alone does not ensure visual reasoning aligned with human expectations. This distinction matters when interpreting a correct answer under prompting: it may show that a model can carry out a counting process, without showing that it will choose that process unaided. These results concern the paper’s benchmark; how far they extend to other visual tasks is unresolved. ## Original source arXiv cs.AI · Article Original source date: Oct 26, 2026 (2026-10-26). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-12T14:31:55.723Z. First observed: 2026-10-28T11:58:40.196Z. Retrieved: 2026-11-12T14:03:04.542Z. - [DecoyCount: Evaluating Visual-Reasoning Misalignment in Counting](https://arxiv.org/abs/2610.34855) ## Limits The reported accuracy and reasoning-alignment gaps are results on the DecoyCount benchmark; the supplied abstract does not establish how they generalize to other visual tasks. The supplied evidence is the paper’s publisher-page abstract and does not provide further methodological detail or results beyond its summary. ## Open question Would the reported reasoning-alignment gaps persist on visual tasks beyond the counting problems in DecoyCount? ## Source-stated The authors introduce DecoyCount, a benchmark using real-world images with designed decoys and paired prompts to evaluate visual counting. > We introduce DecoyCount, a benchmark built from 1,586 real-world images with designed decoys across seven mechanisms - [Supporting source](https://arxiv.org/abs/2610.34855) ## Source-stated The authors define the reasoning-alignment gap as the accuracy difference between cue-guided and unaided counting. > we term this difference in accuracy the reasoning-alignment gap - [Supporting source](https://arxiv.org/abs/2610.34855) ## Source-stated The authors report frontier models scored below 50% accuracy, with reasoning-alignment gaps from 12.1 to 22.1 percentage points on their benchmark. > frontier models remain below 50% accuracy on our benchmark, while alignment gaps range from 12.1-22.1 percentage points - [Supporting source](https://arxiv.org/abs/2610.34855) ## Source-stated The authors report that greater model capability, larger thinking budgets and agentic processes often induce larger gaps. > more capable models, greater thinking budgets, and agentic processes often induce larger gaps - [Supporting source](https://arxiv.org/abs/2610.34855) ## Source-stated The authors report that added scenario context without explicit hinting did not improve aggregate performance. > this added context does not improve aggregate performance - [Supporting source](https://arxiv.org/abs/2610.34855) ## Source-stated The authors conclude that stronger general reasoning does not by itself generalize to visual reasoning aligned with human expectations. > stronger general reasoning does not by itself generalize to aligned visual reasoning - [Supporting source](https://arxiv.org/abs/2610.34855) ## Source and contributor context Qiuyi, Zhang, Xu, Zhen, Gu, Jing, Bang, Seojin, Das, Arnav Mohanty, Hu, Yaning, Kumar, Gautam, Lichtarge, Jared, Xue, Le, Catoto, Bettina, Datta, Akul, Luo, Daniel, Khan, Farha, Valentine, Marcella, Yang, Yinfei, Dai, Andrew M. · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## Frontier model One of the most capable AI models available at a given time; the set shifts as models improve. - [Definition reference](https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper) ## Agentic AI AI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer. - [Definition reference](https://developers.google.com/machine-learning/glossary#agentic) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 6b144c26-5746-49a1-8cd7-f718149c2a1f. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-12T14:31:55.723Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 4ed8fdda52bb80c188646d543a532fdcf6a3cd4bce0b2bc0f03f0ccb52d81d8e. Source text hash: 0e87d8e209ee57c5f19c200e5797bc72b3b5caa02628dc6e5f4c42f203b7aee9. ### passed — 2026-11-12T14:31:55.723Z — anthropic/claude-opus-5.5 Publication: 6b144c26-5746-49a1-8cd7-f718149c2a1f attribution — pass: The draft attributes the paper to Qiuyi (Richard) Zhang and coauthors and identifies it as an arXiv cs.AI paper, matching the author list and listing. Findings and conclusions remain framed as the authors' reports rather than established fact. > Authors:Qiuyi (Richard)Zhang, Zhen Xu, Jing Gu summary — pass: The summary accurately conveys the benchmark design: real-world images with designed decoys, neutral versus cue-guided prompts, and a scenario condition. It also defines the reasoning-alignment gap and reports the headline results and conclusion. The title and question follow from the evidence. Terms such as thinking budgets and agentic processes are kept exactly as technical terms and suit tooltips. The central idea is readable: models can count when cued but fail to choose that process unaided. > models can execute the intended counting process when cue-guided but fail to select it unaided date — pass: originalPublished is 2026-10-26, which matches the v1 submission date. The 5 Nov 2026 date is a later revision, and the 9 Nov 2026 date is the feed entry, so neither should be displayed as the original publication date. > Submitted on 26 Oct 2026 (v1), last revised 5 Nov 2026 support — pass: Every claim is supported verbatim by the abstract: the sub-50% accuracy, the 12.1–22.1 point gaps, larger gaps with more capability and thinking, no gain from added context, and the conclusion. The interpretive sentence about correct answers under prompting closely tracks the source's cue-guided versus unaided finding. The draft scopes the results to the benchmark and leaves generalization as an open question, without unsupported benefits or costs. > this added context does not improve aggregate performance media — pass: The selected media is the verified direct link to the arXiv abstract page, which is the original source. No other source media was found. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/9ae65e60c9d5ee05a08f6388063ba62ba6ed672ba73106de317f3d95f18619cf/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-paper-decoycount-finds-a-gap-between-cued-and-9ae65e60.json)