# arXiv: DeliverBench: Benchmarking AI Agents on Delivering Realistic Multi-Stage Professional Projects [Canonical reading](https://escapemeridian.com/readings/arxiv-deliverbench-benchmarking-ai-agents-on-delivering-137c46a1) The authors introduce DeliverBench to test a claim about professional work: useful delivery means completing a coherent project, not just performing isolated tasks. They argue that errors can carry forward as a project develops, so task-level capability may not translate into a sound final package. The benchmark covers 11 projects in financial due diligence, legal contract drafting, and drug safety reporting. Reconstructed with domain experts, the projects chain subtasks across rounds of new materials and client feedback; expert rubrics score intermediate checkpoints and final delivery. The authors say agents receive no reference outputs, so they carry their own errors forward. Across seven model-harness systems, the best scores were 65.0 out of 100 for final delivery and a 50.0% pass rate for subtasks at a 90-point threshold. In one audit project, all seven systems addressed review findings, but only one preserved traceability. The authors present this as an example of a broader challenge: carrying work forward coherently, not merely answering each task. The abstract does not identify the systems or provide enough detail to assess how representative the projects and scores are. How do the systems perform across the full set of projects, and what kinds of errors most often break continuity between tasks? Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 2, 2026 (2026-11-02). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-10T07:22:53.735Z. First observed: 2026-11-03T09:04:19.583Z. Retrieved: 2026-11-04T03:01:12.736Z. - [DeliverBench: Benchmarking AI Agents on Delivering Realistic Multi-Stage Professional Projects](https://arxiv.org/abs/2611.03412) ## Limits The abstract does not identify the seven model-harness systems or give details sufficient to assess how representative the projects and reported scores are. The reported audit example concerns one project; the abstract does not say whether similar traceability failures occurred elsewhere. ## Open question How do the systems perform across the full set of projects, and what kinds of errors most often break continuity between tasks? ## Source-stated The authors introduce DeliverBench to evaluate whether agents can complete multi-stage projects from brief to final delivery. > we introduce DeliverBench, which tests whether agents can carry a project from brief to final delivery. - [Supporting source](https://arxiv.org/abs/2611.03412) ## Source-stated DeliverBench includes 11 projects across three professional domains, reconstructed with 13 domain experts. > Its 11 projects, spanning financial due diligence, legal contract drafting, and drug safety reporting, are reconstructed with 13 domain experts - [Supporting source](https://arxiv.org/abs/2611.03412) ## Source-stated The projects chain 150 subtasks through 2–15 rounds of new materials and client feedback. > chain 150 subtasks through 2-15 rounds of new materials and client feedback. - [Supporting source](https://arxiv.org/abs/2611.03412) ## Source-stated The abstract reports that the best of seven model-harness systems scored 65.0 out of 100 on final delivery and passed 50.0% of subtasks at a 90-point threshold. > even the best system scores only 65.0 out of 100 on final delivery and passes 50.0% of subtasks at a 90-point threshold. - [Supporting source](https://arxiv.org/abs/2611.03412) ## Source-stated In one audit project, all seven systems addressed review findings, but only one preserved traceability. > in one audit project, all seven systems addressed the review findings, but only one preserved traceability. - [Supporting source](https://arxiv.org/abs/2611.03412) ## Source-stated The authors argue that task-level capability does not necessarily add up to successful project delivery, which also depends on keeping work coherent as it evolves. > task-level capability does not simply compose into delivery, which also depends on keeping the work coherent as it evolves. - [Supporting source](https://arxiv.org/abs/2611.03412) ## Source and contributor context Liu, Yumou, Chen, Mingju, Wen, Zhoufutu, Yang, Qianyu, Wang, Zijie, Liu, Yeqian, Yang, Chenghao, Duan, Tiliang, Cai, Qiyu, Xiao, Yanglihong, Dong, Jiayun, Ma, Jianlan, Miao, Shenhao, Cao, Yuhan, Xu, Lu, Shen, Que, Gao, Tian, Zhang, Ge, Jiao, Jianpeng, Zhou, Xuanhe, Wu, Fan · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 1e53aeb9-84ae-43fe-8c88-bcf9131f3109. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T07:22:53.735Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: afa1da167030bf874f0e9e0d3f4e5c193d42b7d5b947953d4d3afe8742ac42ed. Source text hash: aa77a6af1eccf7a31a99625e13f1864d800d53572f1622be8f68775c88ca955b. ## Revision and source-check history ### Version 1 — 2026-11-10T07:22:53.735Z The authors introduce DeliverBench to test a claim about professional work: useful delivery means completing a coherent project, not just performing isolated tasks. They argue that errors can carry forward as a project develops, so task-level capability may not translate into a sound final package. The benchmark covers 11 projects in financial due diligence, legal contract drafting, and drug safety reporting. Reconstructed with domain experts, the projects chain subtasks across rounds of new materials and client feedback; expert rubrics score intermediate checkpoints and final delivery. The authors say agents receive no reference outputs, so they carry their own errors forward. Across seven model-harness systems, the best scores were 65.0 out of 100 for final delivery and a 50.0% pass rate for subtasks at a 90-point threshold. In one audit project, all seven systems addressed review findings, but only one preserved traceability. The authors present this as an example of a broader challenge: carrying work forward coherently, not merely answering each task. The abstract does not identify the systems or provide enough detail to assess how representative the projects and scores are. How do the systems perform across the full set of projects, and what kinds of errors most often break continuity between tasks? ## Original source arXiv cs.AI · Article Original source date: Nov 2, 2026 (2026-11-02). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-10T07:22:53.735Z. First observed: 2026-11-03T09:04:19.583Z. Retrieved: 2026-11-04T03:01:12.736Z. - [DeliverBench: Benchmarking AI Agents on Delivering Realistic Multi-Stage Professional Projects](https://arxiv.org/abs/2611.03412) ## Limits The abstract does not identify the seven model-harness systems or give details sufficient to assess how representative the projects and reported scores are. The reported audit example concerns one project; the abstract does not say whether similar traceability failures occurred elsewhere. ## Open question How do the systems perform across the full set of projects, and what kinds of errors most often break continuity between tasks? ## Source-stated The authors introduce DeliverBench to evaluate whether agents can complete multi-stage projects from brief to final delivery. > we introduce DeliverBench, which tests whether agents can carry a project from brief to final delivery. - [Supporting source](https://arxiv.org/abs/2611.03412) ## Source-stated DeliverBench includes 11 projects across three professional domains, reconstructed with 13 domain experts. > Its 11 projects, spanning financial due diligence, legal contract drafting, and drug safety reporting, are reconstructed with 13 domain experts - [Supporting source](https://arxiv.org/abs/2611.03412) ## Source-stated The projects chain 150 subtasks through 2–15 rounds of new materials and client feedback. > chain 150 subtasks through 2-15 rounds of new materials and client feedback. - [Supporting source](https://arxiv.org/abs/2611.03412) ## Source-stated The abstract reports that the best of seven model-harness systems scored 65.0 out of 100 on final delivery and passed 50.0% of subtasks at a 90-point threshold. > even the best system scores only 65.0 out of 100 on final delivery and passes 50.0% of subtasks at a 90-point threshold. - [Supporting source](https://arxiv.org/abs/2611.03412) ## Source-stated In one audit project, all seven systems addressed review findings, but only one preserved traceability. > in one audit project, all seven systems addressed the review findings, but only one preserved traceability. - [Supporting source](https://arxiv.org/abs/2611.03412) ## Source-stated The authors argue that task-level capability does not necessarily add up to successful project delivery, which also depends on keeping work coherent as it evolves. > task-level capability does not simply compose into delivery, which also depends on keeping the work coherent as it evolves. - [Supporting source](https://arxiv.org/abs/2611.03412) ## Source and contributor context Liu, Yumou, Chen, Mingju, Wen, Zhoufutu, Yang, Qianyu, Wang, Zijie, Liu, Yeqian, Yang, Chenghao, Duan, Tiliang, Cai, Qiyu, Xiao, Yanglihong, Dong, Jiayun, Ma, Jianlan, Miao, Shenhao, Cao, Yuhan, Xu, Lu, Shen, Que, Gao, Tian, Zhang, Ge, Jiao, Jianpeng, Zhou, Xuanhe, Wu, Fan · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 1e53aeb9-84ae-43fe-8c88-bcf9131f3109. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T07:22:53.735Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: afa1da167030bf874f0e9e0d3f4e5c193d42b7d5b947953d4d3afe8742ac42ed. Source text hash: aa77a6af1eccf7a31a99625e13f1864d800d53572f1622be8f68775c88ca955b. ### uncertain — 2026-11-10T07:22:53.735Z — anthropic/claude-opus-5.5 Publication: 1e53aeb9-84ae-43fe-8c88-bcf9131f3109 attribution — pass: The reading attributes DeliverBench, its design, and its interpretation to the paper's authors on arXiv. It frames results as reported in the abstract, not as established fact. > To address this gap, we introduce DeliverBench, which tests whether agents can carry a project from brief to final delivery. summary — uncertain: The reading is faithful and keeps the authors' central argument. However, the central terms 'model-harness systems' (the evaluated units) and 'traceability' (the key failure in the audit example) are never explained in plain language, and 'agents' is not glossed as LLM-based AI agents. A reader without a specialist background may not grasp what was tested or what failed. Smallest repair: briefly define these terms or add tooltips. Minor fidelity point: 'the best scores were 65.0... and a 50.0% pass rate' could imply two different systems. The source attributes both results to the single best system and frames them with 'even... only'. > Across seven model-harness systems, even the best system scores only 65.0 out of 100 on final delivery and passes 50.0% of subtasks at a 90-point threshold. date — pass: originalPublished 2026-11-02 matches the arXiv v1 submission date. The later feed date of Nov 3 is correctly not used. > \[Submitted on 2 Nov 2026\] support — pass: Every claim matches the abstract: 11 projects, 13 experts, 150 subtasks, 2-15 rounds, no reference outputs, the scores, the audit example, and the conclusion. The limitations and the question follow from what the abstract omits and add no unsupported benefits or costs. > Because no reference outputs are injected, agents inherit their own errors from one task to the next. media — pass: The selected media is the verified arXiv abstract page for this paper, which is the direct original source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/137c46a10825754060d133aa4eeff8b7a63591f69c0282029dcd71484431ac2f/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-deliverbench-benchmarking-ai-agents-on-delivering-137c46a1.json)