# arXiv cs.AI: Decide, Ask, or Defer: Clinical LLMs under Incomplete Evidence [Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-decide-ask-or-defer-clinical-llms-under-incomplete-b8003997) Authors Mingzhan Yang and Weili Wu argue that evaluating clinical LLMs with only “DECIDE” or “ABSTAIN” misses an important distinction: a system may seek more information, or defer a decision to a clinician. They introduce a DECIDE/ASK/DEFER formulation and a blinded protocol that prevents models from using evidence completeness metadata, then evaluate Qwen, Gemini, and GPT on 200 matched clinical evidence states constructed from DDXPlus. Under identical evidence, the models disagree on which action to take in 137 states. In a matched analysis of Qwen, targeted rather than random information changes the likelihood of a later autonomous decision more clearly than diagnostic correctness. Comparing the three-action policy with a DECIDE/ABSTAIN baseline, the authors report a safety autonomy tradeoff: the three-action policy rescues some erroneous autonomous decisions, but also removes some correct ones. Their point is not that adding an “ASK” option uniformly improves clinical decisions; rather, separating information acquisition from deferral makes behavior visible that binary abstention hides, without producing a uniformly improved policy. The abstract reports model behavior on constructed evidence states, not clinical outcomes in practice. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Oct 3, 2026 (2026-10-03). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-12T02:47:47.473Z. First observed: 2026-10-06T07:29:06.180Z. Retrieved: 2026-11-12T02:47:00.198Z. - [Decide, Ask, or Defer: Clinical LLMs under Incomplete Evidence](https://arxiv.org/abs/2610.04542) ## Limits The supplied evidence is the paper’s abstract and does not provide details of the protocol, results by model, or the construction of the 200 DDXPlus evidence states. The abstract describes an evaluation on constructed clinical evidence states; it does not report clinical outcomes in practice. The targeted-versus-random information analysis described in the abstract is for Qwen, so its result should not be generalized to Gemini or GPT. ## Open question Do these action patterns hold beyond the 200 matched clinical evidence states constructed from DDXPlus? ## Source-stated Yang and Wu propose evaluating clinical LLMs with three actions—DECIDE, ASK, and DEFER—rather than merging distinct non-decision states into binary abstention. > Binary DECIDE/ABSTAIN formulations merge distinct non decision states and do not explicitly evaluate information acquisition. - [Supporting source](https://arxiv.org/abs/2610.04542) ## Source-stated The authors evaluated Qwen, Gemini, and GPT on 200 matched clinical evidence states constructed from DDXPlus; the models disagreed on action selection in 137 states. > The models show substantial differences in action selection under identical evidence, with disagreement in 137 of 200 states. - [Supporting source](https://arxiv.org/abs/2610.04542) ## Source-stated In a matched targeted-versus-random analysis of Qwen, the authors say the selected information changed the likelihood of a subsequent autonomous decision more clearly than diagnostic correctness. > selected information changes the likelihood of a subsequent autonomous decision more clearly than diagnostic correctness. - [Supporting source](https://arxiv.org/abs/2610.04542) ## Source-stated The authors report that the three-action policy both rescued some erroneous autonomous decisions and removed some correct ones, and did not yield a uniformly improved decision policy. > deci sions. These results show that separating information acquisition from clinician deferral exposes behavior that binary abstention hides, without yielding a uniformly improved decision policy. - [Supporting source](https://arxiv.org/abs/2610.04542) ## Source and contributor context Yang, Mingzhan, Wu, Weili · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 9a5e8138-fcf3-4484-830f-19048f111b60. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-12T02:47:47.473Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 4242cbd716544f7f3874bd28d2d56f3b3008932f3656c9825d3b86a50624a144. Source text hash: 3ade2a92e0469906571e291285c8be8dadce1e5c6ec852d424ee003712d0ecc6. ## Revision and source-check history ### Version 1 — 2026-11-12T02:47:47.473Z Authors Mingzhan Yang and Weili Wu argue that evaluating clinical LLMs with only “DECIDE” or “ABSTAIN” misses an important distinction: a system may seek more information, or defer a decision to a clinician. They introduce a DECIDE/ASK/DEFER formulation and a blinded protocol that prevents models from using evidence completeness metadata, then evaluate Qwen, Gemini, and GPT on 200 matched clinical evidence states constructed from DDXPlus. Under identical evidence, the models disagree on which action to take in 137 states. In a matched analysis of Qwen, targeted rather than random information changes the likelihood of a later autonomous decision more clearly than diagnostic correctness. Comparing the three-action policy with a DECIDE/ABSTAIN baseline, the authors report a safety autonomy tradeoff: the three-action policy rescues some erroneous autonomous decisions, but also removes some correct ones. Their point is not that adding an “ASK” option uniformly improves clinical decisions; rather, separating information acquisition from deferral makes behavior visible that binary abstention hides, without producing a uniformly improved policy. The abstract reports model behavior on constructed evidence states, not clinical outcomes in practice. ## Original source arXiv cs.AI · Article Original source date: Oct 3, 2026 (2026-10-03). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-12T02:47:47.473Z. First observed: 2026-10-06T07:29:06.180Z. Retrieved: 2026-11-12T02:47:00.198Z. - [Decide, Ask, or Defer: Clinical LLMs under Incomplete Evidence](https://arxiv.org/abs/2610.04542) ## Limits The supplied evidence is the paper’s abstract and does not provide details of the protocol, results by model, or the construction of the 200 DDXPlus evidence states. The abstract describes an evaluation on constructed clinical evidence states; it does not report clinical outcomes in practice. The targeted-versus-random information analysis described in the abstract is for Qwen, so its result should not be generalized to Gemini or GPT. ## Open question Do these action patterns hold beyond the 200 matched clinical evidence states constructed from DDXPlus? ## Source-stated Yang and Wu propose evaluating clinical LLMs with three actions—DECIDE, ASK, and DEFER—rather than merging distinct non-decision states into binary abstention. > Binary DECIDE/ABSTAIN formulations merge distinct non decision states and do not explicitly evaluate information acquisition. - [Supporting source](https://arxiv.org/abs/2610.04542) ## Source-stated The authors evaluated Qwen, Gemini, and GPT on 200 matched clinical evidence states constructed from DDXPlus; the models disagreed on action selection in 137 states. > The models show substantial differences in action selection under identical evidence, with disagreement in 137 of 200 states. - [Supporting source](https://arxiv.org/abs/2610.04542) ## Source-stated In a matched targeted-versus-random analysis of Qwen, the authors say the selected information changed the likelihood of a subsequent autonomous decision more clearly than diagnostic correctness. > selected information changes the likelihood of a subsequent autonomous decision more clearly than diagnostic correctness. - [Supporting source](https://arxiv.org/abs/2610.04542) ## Source-stated The authors report that the three-action policy both rescued some erroneous autonomous decisions and removed some correct ones, and did not yield a uniformly improved decision policy. > deci sions. These results show that separating information acquisition from clinician deferral exposes behavior that binary abstention hides, without yielding a uniformly improved decision policy. - [Supporting source](https://arxiv.org/abs/2610.04542) ## Source and contributor context Yang, Mingzhan, Wu, Weili · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 9a5e8138-fcf3-4484-830f-19048f111b60. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-12T02:47:47.473Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 4242cbd716544f7f3874bd28d2d56f3b3008932f3656c9825d3b86a50624a144. Source text hash: 3ade2a92e0469906571e291285c8be8dadce1e5c6ec852d424ee003712d0ecc6. ### uncertain — 2026-11-12T02:47:47.473Z — anthropic/claude-opus-5.5 Publication: 9a5e8138-fcf3-4484-830f-19048f111b60 attribution — pass: The reading correctly attributes the paper to Mingzhan Yang and Weili Wu on arXiv cs.AI. It frames the findings as the authors' reports, not established fact. > Authors:Mingzhan Yang, Weili Wu summary — uncertain: The summary faithfully conveys the argument, the 137/200 disagreement, the Qwen-only analysis, the safety autonomy tradeoff, and the no-uniform-improvement conclusion. However, several central terms go unexplained for a non-specialist: DDXPlus (what the source data is), 'matched clinical evidence states', 'blinded protocol', and 'evidence completeness metadata'. LLM and the model names Qwen, Gemini and GPT are also not glossed. Smallest repair: add term-library tooltips or brief plain glosses for DDXPlus and evidence completeness metadata, and suggest library entries where they are missing. The closing two sentences repeat the uniform-improvement point, but that is a style issue only. > We evaluate Qwen, Gemini, and GPT on 200 matched clinical evidence states constructed from DDXPlus. date — pass: The originalPublished value of 2026-10-03 matches the v1 submission date. It does not use the later v3 revision (9 Nov) or the feed date (11 Nov). > \[Submitted on 3 Oct 2026 (v1), last revised 9 Nov 2026 (this version, v3)\] support — pass: Every claim is source-stated and supported by an exact excerpt. The rescue/remove tradeoff and the denial of uniform improvement match the abstract. The limitations correctly note the Qwen-only scope, the constructed states, and the absence of clinical outcomes. The question follows from the evidence without being leading. There are no invented benefits or costs. > the three action policy rescues some erroneous autonomous decisions but also removes some correct autonomous deci sions. media — pass: The selected media is the verified direct link to the arXiv abstract page, which is the original source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/b800399722ccacdd9a30676192f997f7105aa1693527bf0273d16073bd23bfac/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-decide-ask-or-defer-clinical-llms-under-incomplete-b8003997.json)