Ravenda, Federico, Cesarini, Mirko, Montagnani, Daniele, Karpenko, Volodymyr, Mira, Antonietta, Raballo, Andrea · Nov 10, 2026
arXiv: Authors argue clinical diagnosis tests what LLMs ask, not just infer
Markdown · JSON · Original source
Federico Ravenda and coauthors introduce Hx2Dx ARENA, an interactive in which move through simulated clinical cases by taking a history, examining the patient and requesting tests. They argue that static evaluations—where a case already presents the relevant findings—miss a central part of diagnosis: gathering the evidence in the first place. In their benchmark, every evaluated model loses diagnostic accuracy compared with the static setting, and performance varies considerably across models. The authors say targeted questions that build a more complete clinical picture account for much of both the gap between models and the loss of accuracy. Their key distinction is that LLMs may reason well once relevant data are available while remaining limited in deciding what information to seek. The authors call closing that gap a prerequisite for safe integration into real-world diagnostic workflows. The result makes question-asking, rather than inference alone, a focus for evaluating clinical reasoning; the reported evidence comes from simulated encounters, not real-world deployment.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 10, 2026 (2026-11-10).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-12T04:01:10.069Z. First observed: 2026-11-11T05:09:34.741Z. Retrieved: 2026-11-12T03:50:00.233Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The abstract describes simulated encounters and does not report evaluation in real-world diagnostic workflows; the results therefore do not establish how models perform in clinical deployment. The abstract does not name the evaluated models or provide detailed results for individual models or configurations.
Open question
How well do the simulated encounters represent the inquiry clinicians use in practice?
Source-stated
The authors introduce Hx2Dx ARENA, an interactive benchmark of simulated clinical cases that proceeds from presentation through history taking, examination and diagnostic workup.
Each case is a simulated encounter that proceeds from initial presentation through history taking and physical examination to diagnostic workup.
Source-stated
The authors contrast interactive diagnosis with static evaluations, which provide a case with relevant findings already selected.
Such evaluations measure inference over already selected evidence; the evaluations do not account for the inquiry used to obtain it.
Source-stated
The authors report that every evaluated model has lower diagnostic accuracy in the interactive setting than in the static setting.
Performance varies considerably across models in the interactive setting, and every model loses diagnostic accuracy relative to the static setting
Source-stated
The authors say targeted questions that build a more complete clinical picture account for much of the performance gap and accuracy loss.
diagnosis improves as questions become more targeted and build a more complete clinical picture.
Source-stated
The authors argue that LLMs remain limited in hypothesis-driven information gathering, even when they can reason well with relevant data available.
LLMs thus reason well once the relevant data are available but remain limited in hypothesis-driven information gathering, in knowing what to ask in order to know.
Source-stated
The authors describe closing the information-gathering gap as a prerequisite for safe integration into real-world diagnostic workflows.
Closing this gap is a prerequisite for their safe integration into real-world diagnostic workflows.
Source and contributor context
Ravenda, Federico, Cesarini, Mirko, Montagnani, Daniele, Karpenko, Volodymyr, Mira, Antonietta, Raballo, Andrea · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 906ffc64-7d53-4ec4-8b8d-f82cdf89974c.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-12T04:01:10.069Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: ef61845e71a3370b7fcfc7c19ab69913c99503a9bf4ea73fe2a2cdd59582d7bd. Source text hash: 4487979c3ce234a3b1716bb37e955e0552e794da85d2b7dee8693d293e494b2d.
Revision and source-check history
Version 1 · 2026-11-12T04:01:10.069Z
arXiv: Authors argue clinical diagnosis tests what LLMs ask, not just infer
Federico Ravenda and coauthors introduce Hx2Dx ARENA, an interactive in which move through simulated clinical cases by taking a history, examining the patient and requesting tests. They argue that static evaluations—where a case already presents the relevant findings—miss a central part of diagnosis: gathering the evidence in the first place. In their benchmark, every evaluated model loses diagnostic accuracy compared with the static setting, and performance varies considerably across models. The authors say targeted questions that build a more complete clinical picture account for much of both the gap between models and the loss of accuracy. Their key distinction is that LLMs may reason well once relevant data are available while remaining limited in deciding what information to seek. The authors call closing that gap a prerequisite for safe integration into real-world diagnostic workflows. The result makes question-asking, rather than inference alone, a focus for evaluating clinical reasoning; the reported evidence comes from simulated encounters, not real-world deployment.
Original source
cs.AI · Article
Source details
Original source date: Nov 10, 2026 (2026-11-10).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-12T04:01:10.069Z. First observed: 2026-11-11T05:09:34.741Z. Retrieved: 2026-11-12T03:50:00.233Z.
Limits
The abstract describes simulated encounters and does not report evaluation in real-world diagnostic workflows; the results therefore do not establish how models perform in clinical deployment. The abstract does not name the evaluated models or provide detailed results for individual models or configurations.
Open question
How well do the simulated encounters represent the inquiry clinicians use in practice?
Source-stated
The authors introduce Hx2Dx ARENA, an interactive benchmark of simulated clinical cases that proceeds from presentation through history taking, examination and diagnostic workup.
Each case is a simulated encounter that proceeds from initial presentation through history taking and physical examination to diagnostic workup.
Source-stated
The authors contrast interactive diagnosis with static evaluations, which provide a case with relevant findings already selected.
Such evaluations measure inference over already selected evidence; the evaluations do not account for the inquiry used to obtain it.
Source-stated
The authors report that every evaluated model has lower diagnostic accuracy in the interactive setting than in the static setting.
Performance varies considerably across models in the interactive setting, and every model loses diagnostic accuracy relative to the static setting
Source-stated
The authors say targeted questions that build a more complete clinical picture account for much of the performance gap and accuracy loss.
diagnosis improves as questions become more targeted and build a more complete clinical picture.
Source-stated
The authors argue that LLMs remain limited in hypothesis-driven information gathering, even when they can reason well with relevant data available.
LLMs thus reason well once the relevant data are available but remain limited in hypothesis-driven information gathering, in knowing what to ask in order to know.
Source-stated
The authors describe closing the information-gathering gap as a prerequisite for safe integration into real-world diagnostic workflows.
Closing this gap is a prerequisite for their safe integration into real-world diagnostic workflows.
Source and contributor context
Ravenda, Federico, Cesarini, Mirko, Montagnani, Daniele, Karpenko, Volodymyr, Mira, Antonietta, Raballo, Andrea · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 906ffc64-7d53-4ec4-8b8d-f82cdf89974c.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-12T04:01:10.069Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: ef61845e71a3370b7fcfc7c19ab69913c99503a9bf4ea73fe2a2cdd59582d7bd. Source text hash: 4487979c3ce234a3b1716bb37e955e0552e794da85d2b7dee8693d293e494b2d.
passed · 2026-11-12T04:01:10.069Z · anthropic/claude-opus-5.5
Publication: 906ffc64-7d53-4ec4-8b8d-f82cdf89974c
attribution — pass: The reading correctly credits Federico Ravenda and coauthors for introducing Hx2Dx ARENA. The arguments about static evaluations, lost accuracy and information gathering remain attributed to the authors rather than presented as established fact.
Authors:Federico Ravenda, Mirko Cesarini, Daniele Montagnani, Volodymyr Karpenko, Antonietta Mira, Andrea Raballo
summary — pass: A general reader can follow the summary. It explains static versus interactive evaluation and the simulated steps of history taking, examination and tests, and it states the central thesis. Minor: 'may reason well' slightly softens the source's direct 'LLMs thus reason well'; dropping 'may' would restore the authors' force. It also omits the 1,000 NEJM cases. Neither prevents understanding. The question is a genuine open issue about how well the simulation represents real practice.
LLMs thus reason well once the relevant data are available but remain limited in hypothesis-driven information gathering, in knowing what to ask in order to know.
date — pass: The originalPublished date, 2026-11-10, matches the arXiv v1 submission date. It is not the later feed date.
[Submitted on 10 Nov 2026]
support — pass: Each claim matches the abstract. The excerpt for claim 4 shows the improvement trend, while the 'accounts for much' wording comes from the adjacent sentence; it is still supported. The closing framing about simulated rather than deployed evidence and the limitations are grounded and conditional. No benefits or costs are invented.
The ability to ask the right questions accounts for much of both the gap between models and the loss of accuracy
media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA language model with many learned numerical settings, used to process and generate language.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA language model with many learned numerical settings, used to process and generate language.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference