He, Xiangheng, Zhao, Mingjun, Fan, Zhehao, Li, Zhengjia, Sun, Xiaopeng, Xiao, Qingqing, Chen, Haolan, Xu, Yu · Nov 9, 2026
arXiv cs.AI: Joi-Bench-Task tests coordination in full-duplex voice agents
Markdown · JSON · Original source
The authors of Joi-Bench-Task introduce a for a problem they say existing voice-agent benchmarks do not isolate: coordinating task execution with a conversation that can change while the agent is speaking. Full-duplex voice agents let users refine requests, redirect ongoing work, or seek clarification without waiting for the agent to finish. The benchmark tracks what an agent is executing, has completed, communicated, and still needs to report as the conversation evolves. Its interactive user simulator introduces configurable interventions during tool execution and spoken result delivery, allowing controlled, reproducible evaluation. In evaluations of five leading commercial voice agents, the authors report persistent coordination failures. Under request changes, the highest result completion rate was 55%; after interruptions, four agents had average information omission rates of at least 23%. The authors argue these results make task coordination a central challenge for voice agents in complex, evolving conversations. The abstract reports benchmark results, not whether performance on this benchmark predicts how agents will handle voice interactions outside the evaluation.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 9, 2026 (2026-11-09).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-12T06:04:51.281Z. First observed: 2026-11-10T08:39:47.717Z. Retrieved: 2026-11-12T06:03:47.121Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The evidence is the paper’s abstract; it does not name the five agents or give the evaluation details behind the reported rates. The abstract describes configurable intervention timing but does not report how results vary across those timings. The abstract reports results but does not establish how they relate to voice-agent performance outside the evaluation.
Open question
How do the reported coordination results vary across the simulator’s intervention timings?
Source-stated
The authors introduce Joi-Bench-Task to assess coordination between task execution and conversation as that conversation evolves.
We introduce a benchmark that assesses this coordination by examining whether agents track what is being executed, what has been completed, what has been communicated, and what remains to be reported as the conversation evolves.
Source-stated
The benchmark’s interactive user simulator introduces configurable interventions during tool execution and spoken result delivery.
We propose an interactive user simulator that introduces configurable interventions at designated timing in tool execution and spoken result delivery, supporting controllable and reproducible evaluation conditions.
Source-stated
In evaluations of five commercial voice agents, the highest result completion rate under request changes was 55%.
Under request changes, the highest result completion rate is only 55%, indicating difficulty for current voice agents adapting to new requests while preserving earlier obligations.
Source-stated
After user interruption, four of the five evaluated agents had average information omission rates of at least 23%.
After user interruption, four of the five agents exhibit average information omission rates of at least 23%.
Source-stated
The authors characterize task coordination as a central challenge for voice agents in complex, evolving conversations.
These findings highlight task coordination as a central challenge for current voice agents in complex, evolving conversations.
Source and contributor context
He, Xiangheng, Zhao, Mingjun, Fan, Zhehao, Li, Zhengjia, Sun, Xiaopeng, Xiao, Qingqing, Chen, Haolan, Xu, Yu · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 705e2a49-c0ab-4743-8a66-ea22cf9c2370.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-12T06:04:51.281Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: b2af5e99018ac67ef9bf0842d34692628c2e677c55b5ef06d1066ce51badbd1f. Source text hash: 423abaaaa14d99b9e045299b8099fce898d6c1ebbb5e07d7221107897c434457.
Revision and source-check history
Version 1 · 2026-11-12T06:04:51.281Z
arXiv cs.AI: Joi-Bench-Task tests coordination in full-duplex voice agents
The authors of Joi-Bench-Task introduce a for a problem they say existing voice-agent benchmarks do not isolate: coordinating task execution with a conversation that can change while the agent is speaking. Full-duplex voice agents let users refine requests, redirect ongoing work, or seek clarification without waiting for the agent to finish. The benchmark tracks what an agent is executing, has completed, communicated, and still needs to report as the conversation evolves. Its interactive user simulator introduces configurable interventions during tool execution and spoken result delivery, allowing controlled, reproducible evaluation. In evaluations of five leading commercial voice agents, the authors report persistent coordination failures. Under request changes, the highest result completion rate was 55%; after interruptions, four agents had average information omission rates of at least 23%. The authors argue these results make task coordination a central challenge for voice agents in complex, evolving conversations. The abstract reports benchmark results, not whether performance on this benchmark predicts how agents will handle voice interactions outside the evaluation.
Original source
cs.AI · Article
Source details
Original source date: Nov 9, 2026 (2026-11-09).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-12T06:04:51.281Z. First observed: 2026-11-10T08:39:47.717Z. Retrieved: 2026-11-12T06:03:47.121Z.
Limits
The evidence is the paper’s abstract; it does not name the five agents or give the evaluation details behind the reported rates. The abstract describes configurable intervention timing but does not report how results vary across those timings. The abstract reports results but does not establish how they relate to voice-agent performance outside the evaluation.
Open question
How do the reported coordination results vary across the simulator’s intervention timings?
Source-stated
The authors introduce Joi-Bench-Task to assess coordination between task execution and conversation as that conversation evolves.
We introduce a benchmark that assesses this coordination by examining whether agents track what is being executed, what has been completed, what has been communicated, and what remains to be reported as the conversation evolves.
Source-stated
The benchmark’s interactive user simulator introduces configurable interventions during tool execution and spoken result delivery.
We propose an interactive user simulator that introduces configurable interventions at designated timing in tool execution and spoken result delivery, supporting controllable and reproducible evaluation conditions.
Source-stated
In evaluations of five commercial voice agents, the highest result completion rate under request changes was 55%.
Under request changes, the highest result completion rate is only 55%, indicating difficulty for current voice agents adapting to new requests while preserving earlier obligations.
Source-stated
After user interruption, four of the five evaluated agents had average information omission rates of at least 23%.
After user interruption, four of the five agents exhibit average information omission rates of at least 23%.
Source-stated
The authors characterize task coordination as a central challenge for voice agents in complex, evolving conversations.
These findings highlight task coordination as a central challenge for current voice agents in complex, evolving conversations.
Source and contributor context
He, Xiangheng, Zhao, Mingjun, Fan, Zhehao, Li, Zhengjia, Sun, Xiaopeng, Xiao, Qingqing, Chen, Haolan, Xu, Yu · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 705e2a49-c0ab-4743-8a66-ea22cf9c2370.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-12T06:04:51.281Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: b2af5e99018ac67ef9bf0842d34692628c2e677c55b5ef06d1066ce51badbd1f. Source text hash: 423abaaaa14d99b9e045299b8099fce898d6c1ebbb5e07d7221107897c434457.
uncertain · 2026-11-12T06:04:51.281Z · anthropic/claude-opus-5.5
Publication: 705e2a49-c0ab-4743-8a66-ea22cf9c2370
attribution — pass: The draft correctly attributes the benchmark, simulator, evaluation results and the conclusion to the paper's authors on arXiv cs.AI. It does not present the claims as established fact or as an Escape Meridian opinion.
Authors:Xiangheng He, Mingjun Zhao, Zhehao Fan, Zhengjia Li, Xiaopeng Sun, Qingqing Xiao, Haolan Chen, Yu Xu
summary — uncertain: The draft is clear overall and explains full-duplex accessibly, but it has small scope shifts. First, the source says existing benchmarks 'insufficiently isolate' coordination; the draft strengthens this to 'do not isolate'. Repair: write 'do not sufficiently isolate'. Second, the conclusion drops 'current'. Repair: write 'current voice agents'. Third, it drops the authors' 'only 55%' and their reading of that score: difficulty adapting to new requests while preserving earlier obligations. Repair: restore that clause. Finally, 'result completion rate' and 'information omission rate' are not explained; the authors' adaptation clause would help.
existing voice-agent benchmarks insufficiently isolate coordination between task execution and conversation
date — pass: The originalPublished date of 2026-11-09 matches the arXiv v1 submission date. The feed date of Nov 10 is correctly not used.
[Submitted on 9 Nov 2026]
support — pass: Each claim is directly supported by exact abstract excerpts. The closing caveat about real-world relevance is appropriately framed as unknown. The question about how results vary with intervention timing follows from the configurable-timing design, and the limitations acknowledge that the abstract does not report it. No unsupported benefits or costs are implied.
Under request changes, the highest result completion rate is only 55%
media — pass: The selected media is the verified arXiv abstract page for this paper, the direct original source. No better media candidate was supplied.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference