arXiv cs.AI: Decide, Ask, or Defer: Clinical LLMs under Incomplete Evidence

arXiv cs.AI: Decide, Ask, or Defer: Clinical LLMs under Incomplete Evidence

Authors Mingzhan Yang and Weili Wu argue that evaluating clinical LLMs with only “DECIDE” or “ABSTAIN” misses an important distinction: a system may seek more information, or defer a decision to a clinician. They introduce a DECIDE/ASK/DEFER formulation and a blinded protocol that prevents models from using evidence completeness metadata, then evaluate Qwen, Gemini, and GPT on 200 matched clinical evidence states constructed from DDXPlus. Under identical evidence, the models disagree on which action to take in 137 states. In a matched analysis of Qwen, targeted rather than random information changes the likelihood of a later autonomous decision more clearly than diagnostic correctness. Comparing the three-action policy with a DECIDE/ABSTAIN baseline, the authors report a safety autonomy tradeoff: the three-action policy rescues some erroneous autonomous decisions, but also removes some correct ones. Their point is not that adding an “ASK” option uniformly improves clinical decisions; rather, separating information acquisition from deferral makes behavior visible that binary abstention hides, without producing a uniformly improved policy. The abstract reports model behavior on constructed evidence states, not clinical outcomes in practice.

Publication: 9a5e8138-fcf3-4484-830f-19048f111b60

attribution — pass: The reading correctly attributes the paper to Mingzhan Yang and Weili Wu on arXiv cs.AI. It frames the findings as the authors' reports, not established fact.

Authors:Mingzhan Yang, Weili Wu

summary — uncertain: The summary faithfully conveys the argument, the 137/200 disagreement, the Qwen-only analysis, the safety autonomy tradeoff, and the no-uniform-improvement conclusion. However, several central terms go unexplained for a non-specialist: DDXPlus (what the source data is), 'matched clinical evidence states', 'blinded protocol', and 'evidence completeness metadata'. LLM and the model names Qwen, Gemini and GPT are also not glossed. Smallest repair: add term-library tooltips or brief plain glosses for DDXPlus and evidence completeness metadata, and suggest library entries where they are missing. The closing two sentences repeat the uniform-improvement point, but that is a style issue only.

We evaluate Qwen, Gemini, and GPT on 200 matched clinical evidence states constructed from DDXPlus.

date — pass: The originalPublished value of 2026-10-03 matches the v1 submission date. It does not use the later v3 revision (9 Nov) or the feed date (11 Nov).

[Submitted on 3 Oct 2026 (v1), last revised 9 Nov 2026 (this version, v3)]

support — pass: Every claim is source-stated and supported by an exact excerpt. The rescue/remove tradeoff and the denial of uniform improvement match the abstract. The limitations correctly note the Qwen-only scope, the constructed states, and the absence of clinical outcomes. The question follows from the evidence without being leading. There are no invented benefits or costs.

the three action policy rescues some erroneous autonomous decisions but also removes some correct autonomous deci sions.

media — pass: The selected media is the verified direct link to the arXiv abstract page, which is the original source.