# An arXiv paper tests whether AI agents can search scanned financial archives [Canonical reading](https://escapemeridian.com/readings/an-arxiv-paper-tests-whether-ai-agents-can-search-scanned-cea1a546) The authors of TreasuryQA Bench introduce a benchmark for a practical task: finding figures in scanned financial records. They argue that existing benchmarks leave out important steps, either giving agents the relevant documents or letting them search text already extracted from documents. Their benchmark combines both steps. Agents search 1,435 scanned U.S. Treasury publications through Dropbox's Model Context Protocol (MCP) server, read relevant pages using OCR or their own tools, and return one exact figure. The authors test ten agents; the best solves 64.8% of attempts and the worst 16.7%, which they say leaves the benchmark far from saturated. The results also point to a specific weakness: every agent performs worse on the blurriest scans, and the most common cause of failure is misreading a digit. Escape Meridian interpretation: for organizations hoping to use agents on scanned records, searching for a document is not enough; reading the scan accurately also matters. The benchmark measures performance on its tasks, not whether agents can reliably handle financial records in real-world use. An open question is whether improving scan quality or OCR would reduce digit errors on this benchmark. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Oct 28, 2026 (2026-10-28). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-13T03:32:40.338Z. First observed: 2026-10-30T06:40:37.664Z. Retrieved: 2026-10-31T11:02:48.633Z. - [TreasuryQA Bench: Can AI Agents Work an Archive of Scanned Financial Records?](https://arxiv.org/abs/2610.38924) ## Limits The evidence is an arXiv abstract; it does not provide details needed to assess the full experimental design or results. The reported benchmark results do not establish how agents perform on real-world financial archives. The abstract does not say whether improving scan quality or OCR would reduce errors. ## Open question Would improving scan quality or OCR reduce digit misreads on this benchmark? ## Source-stated The authors introduce a benchmark requiring agents both to search for relevant scanned financial records and to read them. > We introduce TreasuryQA Bench, which requires both: 70 questions over 1,435 scanned U.S. Treasury publications - [Supporting source](https://arxiv.org/abs/2610.38924) ## Source-stated Agents search through Dropbox's MCP server, read needed pages with OCR or their own tools, and return one exact figure. > the agent searches for the relevant file through Dropbox's MCP server, reads the pages it needs with Reducto's hosted OCR server or its own tools, and replies with one exact figure. - [Supporting source](https://arxiv.org/abs/2610.38924) ## Source-stated Across ten agents, the best solved 64.8% of attempts and the worst 16.7%. > The best, Claude Opus 5.5 in Claude Code, solves 64.8% of attempts and the worst, GPT-5.6 Terra in Codex, solves 16.7% - [Supporting source](https://arxiv.org/abs/2610.38924) ## Source-stated All tested agents performed worse on the blurriest scans, and digit misreads were the most common cause of failure. > every agent solves fewer tasks on the blurriest scans. Most failed attempts reply with a wrong number rather than no number, and the most common cause is a digit misread from the scan. - [Supporting source](https://arxiv.org/abs/2610.38924) ## Escape Meridian interpretation The results suggest that reading scan images accurately is a distinct challenge from finding the right document. > In some, the relevant documents are handed to the agent, so it never has to search. In others, the agent searches through documents with pre-extracted text, so it never has to read scanned images. - [Supporting source](https://arxiv.org/abs/2610.38924) ## Source and contributor context Tambi, Ritiz, Shayegan, Sahar, Meyer, Joshua, Kim, Sun, Khan, Ali, Shih, Victor, Jamei, Mehdi, Partovi, Andi · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## AI agent An AI system that chooses and carries out actions toward a goal, often using software tools. - [Definition reference](https://www.ibm.com/think/topics/ai-agents) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Figure Figure develops humanoid robots and publishes its own tests of how they perform physical tasks. - [Explore this context](/entities/figure) - [Reference](https://www.figure.ai/news) - [Reference](https://www.figure.ai/news/helix-2-5-zero-shot-30-home-generalization) ## Editorial provenance Version 1. Publication ID: 320722dc-69e2-4add-84bf-79171c7cf39f. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-13T03:32:40.338Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: a0e1f03e01d4b6deb509c82367cb6b0c4b62c59fa5cf2542ef8765ee8b10bf5f. Source text hash: a9eb2da725d0b7f8e0e7e912dca154378a9fe90f0a2b70e5dfdedb61aafb639b. ## Revision and source-check history ### Version 1 — 2026-11-13T03:32:40.338Z The authors of TreasuryQA Bench introduce a benchmark for a practical task: finding figures in scanned financial records. They argue that existing benchmarks leave out important steps, either giving agents the relevant documents or letting them search text already extracted from documents. Their benchmark combines both steps. Agents search 1,435 scanned U.S. Treasury publications through Dropbox's Model Context Protocol (MCP) server, read relevant pages using OCR or their own tools, and return one exact figure. The authors test ten agents; the best solves 64.8% of attempts and the worst 16.7%, which they say leaves the benchmark far from saturated. The results also point to a specific weakness: every agent performs worse on the blurriest scans, and the most common cause of failure is misreading a digit. Escape Meridian interpretation: for organizations hoping to use agents on scanned records, searching for a document is not enough; reading the scan accurately also matters. The benchmark measures performance on its tasks, not whether agents can reliably handle financial records in real-world use. An open question is whether improving scan quality or OCR would reduce digit errors on this benchmark. ## Original source arXiv cs.AI · Article Original source date: Oct 28, 2026 (2026-10-28). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-13T03:32:40.338Z. First observed: 2026-10-30T06:40:37.664Z. Retrieved: 2026-10-31T11:02:48.633Z. - [TreasuryQA Bench: Can AI Agents Work an Archive of Scanned Financial Records?](https://arxiv.org/abs/2610.38924) ## Limits The evidence is an arXiv abstract; it does not provide details needed to assess the full experimental design or results. The reported benchmark results do not establish how agents perform on real-world financial archives. The abstract does not say whether improving scan quality or OCR would reduce errors. ## Open question Would improving scan quality or OCR reduce digit misreads on this benchmark? ## Source-stated The authors introduce a benchmark requiring agents both to search for relevant scanned financial records and to read them. > We introduce TreasuryQA Bench, which requires both: 70 questions over 1,435 scanned U.S. Treasury publications - [Supporting source](https://arxiv.org/abs/2610.38924) ## Source-stated Agents search through Dropbox's MCP server, read needed pages with OCR or their own tools, and return one exact figure. > the agent searches for the relevant file through Dropbox's MCP server, reads the pages it needs with Reducto's hosted OCR server or its own tools, and replies with one exact figure. - [Supporting source](https://arxiv.org/abs/2610.38924) ## Source-stated Across ten agents, the best solved 64.8% of attempts and the worst 16.7%. > The best, Claude Opus 5.5 in Claude Code, solves 64.8% of attempts and the worst, GPT-5.6 Terra in Codex, solves 16.7% - [Supporting source](https://arxiv.org/abs/2610.38924) ## Source-stated All tested agents performed worse on the blurriest scans, and digit misreads were the most common cause of failure. > every agent solves fewer tasks on the blurriest scans. Most failed attempts reply with a wrong number rather than no number, and the most common cause is a digit misread from the scan. - [Supporting source](https://arxiv.org/abs/2610.38924) ## Escape Meridian interpretation The results suggest that reading scan images accurately is a distinct challenge from finding the right document. > In some, the relevant documents are handed to the agent, so it never has to search. In others, the agent searches through documents with pre-extracted text, so it never has to read scanned images. - [Supporting source](https://arxiv.org/abs/2610.38924) ## Source and contributor context Tambi, Ritiz, Shayegan, Sahar, Meyer, Joshua, Kim, Sun, Khan, Ali, Shih, Victor, Jamei, Mehdi, Partovi, Andi · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## AI agent An AI system that chooses and carries out actions toward a goal, often using software tools. - [Definition reference](https://www.ibm.com/think/topics/ai-agents) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Figure Figure develops humanoid robots and publishes its own tests of how they perform physical tasks. - [Explore this context](/entities/figure) - [Reference](https://www.figure.ai/news) - [Reference](https://www.figure.ai/news/helix-2-5-zero-shot-30-home-generalization) ## Editorial provenance Version 1. Publication ID: 320722dc-69e2-4add-84bf-79171c7cf39f. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-13T03:32:40.338Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: a0e1f03e01d4b6deb509c82367cb6b0c4b62c59fa5cf2542ef8765ee8b10bf5f. Source text hash: a9eb2da725d0b7f8e0e7e912dca154378a9fe90f0a2b70e5dfdedb61aafb639b. ### passed — 2026-11-13T03:32:40.338Z — anthropic/claude-opus-5.5 Publication: 320722dc-69e2-4add-84bf-79171c7cf39f attribution — pass: The reading attributes the benchmark, its framing of existing benchmarks, and the results to the paper's authors. The 'far from saturated' judgment is presented as their claim, and the Escape Meridian inference is labeled. > Authors:Ritiz Tambi, Sahar Shayegan, Joshua Meyer, Sun Kim, Ali Khan, Victor Shih, Mehdi Jamei, Andi Partovi summary — pass: The reading accurately conveys the gap in existing benchmarks, the task design, the score range, the scan-quality effect and digit misreads. A general reader can follow it, and MCP and OCR can be explained by term tooltips. One fidelity point is omitted: failed attempts mostly gave a wrong number rather than no number. A one-clause addition would restore that stake. The question follows from the evidence and is not leading. > Most failed attempts reply with a wrong number rather than no number, and the most common cause is a digit misread from the scan. date — pass: originalPublished is 2026-10-28, which matches the v1 submission date on arXiv. The later feed date is not used. > \[Submitted on 28 Oct 2026\] support — pass: The source-stated claims match the abstract verbatim. The labeled interpretation, that accurate reading matters beyond finding a document, is supported by the digit-misread and blur findings and is appropriately limited. The reading notes that benchmark scores do not establish real-world reliability, and the limitations acknowledge that the evidence is only the abstract. > every agent solves fewer tasks on the blurriest scans media — pass: The selected media is the verified direct link to the original arXiv abstract page. It matches the source URL and is the most direct way to access the paper. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/cea1a5460942d4043423d82244225fa6f40694d18117194c739f46088eff5f32/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/an-arxiv-paper-tests-whether-ai-agents-can-search-scanned-cea1a546.json)