Liu, Yumou, Chen, Mingju, Wen, Zhoufutu, Yang, Qianyu, Wang, Zijie, Liu, Yeqian, Yang, Chenghao, Duan, Tiliang, Cai, Qiyu, Xiao, Yanglihong, Dong, Jiayun, Ma, Jianlan, Miao, Shenhao, Cao, Yuhan, Xu, Lu, Shen, Que, Gao, Tian, Zhang, Ge, Jiao, Jianpeng, Zhou, Xuanhe, Wu, Fan · Nov 2, 2026
arXiv: DeliverBench: Benchmarking AI Agents on Delivering Realistic Multi-Stage Professional Projects
Markdown · JSON · Original source
The authors introduce DeliverBench to test a claim about professional work: useful delivery means completing a coherent project, not just performing isolated tasks. They argue that errors can carry forward as a project develops, so task-level capability may not translate into a sound final package. The covers 11 projects in financial due diligence, legal contract drafting, and drug safety reporting. Reconstructed with domain experts, the projects chain subtasks across rounds of new materials and client feedback; expert rubrics score intermediate checkpoints and final delivery. The authors say agents receive no reference outputs, so they carry their own errors forward. Across seven model-harness systems, the best scores were 65.0 out of 100 for final delivery and a 50.0% pass rate for subtasks at a 90-point threshold. In one audit project, all seven systems addressed review findings, but only one preserved traceability. The authors present this as an example of a broader challenge: carrying work forward coherently, not merely answering each task. The abstract does not identify the systems or provide enough detail to assess how representative the projects and scores are. How do the systems perform across the full set of projects, and what kinds of errors most often break continuity between tasks?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 2, 2026 (2026-11-02).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T07:22:53.735Z. First observed: 2026-11-03T09:04:19.583Z. Retrieved: 2026-11-04T03:01:12.736Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The abstract does not identify the seven model-harness systems or give details sufficient to assess how representative the projects and reported scores are. The reported audit example concerns one project; the abstract does not say whether similar traceability failures occurred elsewhere.
Open question
How do the systems perform across the full set of projects, and what kinds of errors most often break continuity between tasks?
Source-stated
The authors introduce DeliverBench to evaluate whether agents can complete multi-stage projects from brief to final delivery.
we introduce DeliverBench, which tests whether agents can carry a project from brief to final delivery.
Source-stated
DeliverBench includes 11 projects across three professional domains, reconstructed with 13 domain experts.
Its 11 projects, spanning financial due diligence, legal contract drafting, and drug safety reporting, are reconstructed with 13 domain experts
Source-stated
The projects chain 150 subtasks through 2–15 rounds of new materials and client feedback.
chain 150 subtasks through 2-15 rounds of new materials and client feedback.
Source-stated
The abstract reports that the best of seven model-harness systems scored 65.0 out of 100 on final delivery and passed 50.0% of subtasks at a 90-point threshold.
even the best system scores only 65.0 out of 100 on final delivery and passes 50.0% of subtasks at a 90-point threshold.
Source-stated
In one audit project, all seven systems addressed review findings, but only one preserved traceability.
in one audit project, all seven systems addressed the review findings, but only one preserved traceability.
Source-stated
The authors argue that task-level capability does not necessarily add up to successful project delivery, which also depends on keeping work coherent as it evolves.
task-level capability does not simply compose into delivery, which also depends on keeping the work coherent as it evolves.
Source and contributor context
Liu, Yumou, Chen, Mingju, Wen, Zhoufutu, Yang, Qianyu, Wang, Zijie, Liu, Yeqian, Yang, Chenghao, Duan, Tiliang, Cai, Qiyu, Xiao, Yanglihong, Dong, Jiayun, Ma, Jianlan, Miao, Shenhao, Cao, Yuhan, Xu, Lu, Shen, Que, Gao, Tian, Zhang, Ge, Jiao, Jianpeng, Zhou, Xuanhe, Wu, Fan · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 1e53aeb9-84ae-43fe-8c88-bcf9131f3109.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T07:22:53.735Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: afa1da167030bf874f0e9e0d3f4e5c193d42b7d5b947953d4d3afe8742ac42ed. Source text hash: aa77a6af1eccf7a31a99625e13f1864d800d53572f1622be8f68775c88ca955b.
Revision and source-check history
Version 1 · 2026-11-10T07:22:53.735Z
arXiv: DeliverBench: Benchmarking AI Agents on Delivering Realistic Multi-Stage Professional Projects
The authors introduce DeliverBench to test a claim about professional work: useful delivery means completing a coherent project, not just performing isolated tasks. They argue that errors can carry forward as a project develops, so task-level capability may not translate into a sound final package. The covers 11 projects in financial due diligence, legal contract drafting, and drug safety reporting. Reconstructed with domain experts, the projects chain subtasks across rounds of new materials and client feedback; expert rubrics score intermediate checkpoints and final delivery. The authors say agents receive no reference outputs, so they carry their own errors forward. Across seven model-harness systems, the best scores were 65.0 out of 100 for final delivery and a 50.0% pass rate for subtasks at a 90-point threshold. In one audit project, all seven systems addressed review findings, but only one preserved traceability. The authors present this as an example of a broader challenge: carrying work forward coherently, not merely answering each task. The abstract does not identify the systems or provide enough detail to assess how representative the projects and scores are. How do the systems perform across the full set of projects, and what kinds of errors most often break continuity between tasks?
Original source
cs.AI · Article
Source details
Original source date: Nov 2, 2026 (2026-11-02).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T07:22:53.735Z. First observed: 2026-11-03T09:04:19.583Z. Retrieved: 2026-11-04T03:01:12.736Z.
Limits
The abstract does not identify the seven model-harness systems or give details sufficient to assess how representative the projects and reported scores are. The reported audit example concerns one project; the abstract does not say whether similar traceability failures occurred elsewhere.
Open question
How do the systems perform across the full set of projects, and what kinds of errors most often break continuity between tasks?
Source-stated
The authors introduce DeliverBench to evaluate whether agents can complete multi-stage projects from brief to final delivery.
we introduce DeliverBench, which tests whether agents can carry a project from brief to final delivery.
Source-stated
DeliverBench includes 11 projects across three professional domains, reconstructed with 13 domain experts.
Its 11 projects, spanning financial due diligence, legal contract drafting, and drug safety reporting, are reconstructed with 13 domain experts
Source-stated
The projects chain 150 subtasks through 2–15 rounds of new materials and client feedback.
chain 150 subtasks through 2-15 rounds of new materials and client feedback.
Source-stated
The abstract reports that the best of seven model-harness systems scored 65.0 out of 100 on final delivery and passed 50.0% of subtasks at a 90-point threshold.
even the best system scores only 65.0 out of 100 on final delivery and passes 50.0% of subtasks at a 90-point threshold.
Source-stated
In one audit project, all seven systems addressed review findings, but only one preserved traceability.
in one audit project, all seven systems addressed the review findings, but only one preserved traceability.
Source-stated
The authors argue that task-level capability does not necessarily add up to successful project delivery, which also depends on keeping work coherent as it evolves.
task-level capability does not simply compose into delivery, which also depends on keeping the work coherent as it evolves.
Source and contributor context
Liu, Yumou, Chen, Mingju, Wen, Zhoufutu, Yang, Qianyu, Wang, Zijie, Liu, Yeqian, Yang, Chenghao, Duan, Tiliang, Cai, Qiyu, Xiao, Yanglihong, Dong, Jiayun, Ma, Jianlan, Miao, Shenhao, Cao, Yuhan, Xu, Lu, Shen, Que, Gao, Tian, Zhang, Ge, Jiao, Jianpeng, Zhou, Xuanhe, Wu, Fan · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 1e53aeb9-84ae-43fe-8c88-bcf9131f3109.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T07:22:53.735Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: afa1da167030bf874f0e9e0d3f4e5c193d42b7d5b947953d4d3afe8742ac42ed. Source text hash: aa77a6af1eccf7a31a99625e13f1864d800d53572f1622be8f68775c88ca955b.
uncertain · 2026-11-10T07:22:53.735Z · anthropic/claude-opus-5.5
Publication: 1e53aeb9-84ae-43fe-8c88-bcf9131f3109
attribution — pass: The reading attributes DeliverBench, its design, and its interpretation to the paper's authors on arXiv. It frames results as reported in the abstract, not as established fact.
To address this gap, we introduce DeliverBench, which tests whether agents can carry a project from brief to final delivery.
summary — uncertain: The reading is faithful and keeps the authors' central argument. However, the central terms 'model-harness systems' (the evaluated units) and 'traceability' (the key failure in the audit example) are never explained in plain language, and 'agents' is not glossed as LLM-based AI agents. A reader without a specialist background may not grasp what was tested or what failed. Smallest repair: briefly define these terms or add tooltips. Minor fidelity point: 'the best scores were 65.0... and a 50.0% pass rate' could imply two different systems. The source attributes both results to the single best system and frames them with 'even... only'.
Across seven model-harness systems, even the best system scores only 65.0 out of 100 on final delivery and passes 50.0% of subtasks at a 90-point threshold.
date — pass: originalPublished 2026-11-02 matches the arXiv v1 submission date. The later feed date of Nov 3 is correctly not used.
[Submitted on 2 Nov 2026]
support — pass: Every claim matches the abstract: 11 projects, 13 experts, 150 subtasks, 2-15 rounds, no reference outputs, the scores, the audit example, and the conclusion. The limitations and the question follow from what the abstract omits and add no unsupported benefits or costs.
Because no reference outputs are injected, agents inherit their own errors from one task to the next.
media — pass: The selected media is the verified arXiv abstract page for this paper, which is the direct original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv