Ashrafi, Negin, Daneshjou, Roxana · Nov 14, 2026
Researchers characterize when clinical LLMs recommend therapies despite genomic resistance
Markdown · JSON · Original source
Negin Ashrafi and Roxana Daneshjou report that, across ten models tested on 1,464 resistance cases, models endorsed therapies despite genomic evidence of resistance in 21.8–81.4% of cases. But an endorsement rate alone can confuse resistance handling with a general tendency to agree: one endorsed nearly as many sensitivity controls as resistance cases. The authors’ exploratory matched-group analysis found that results depended strongly on the reference knowledge base and evidence strength. When a second source independently confirmed resistance, four models across three vendors endorsed at most 2.4% of cases; at weaker evidence tiers, discrimination was not measurable. The paper also reports that models sometimes acknowledged resistance and still recommended the therapy. Repeating a model’s own resistance conclusion reduced endorsements, while falsely attributing the opposite conclusion raised them; providing correct curated evidence reduced unqualified strong recommendations. That last test used an oracle condition, not retrieval, so it does not establish how well models find the right evidence in practice. The findings describe judgments on curated cases, not treatment outcomes for patients.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 14, 2026 (2026-11-14).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-17T10:32:01.073Z. First observed: 2026-11-17T08:31:07.176Z. Retrieved: 2026-11-17T10:19:34.068Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The authors call the matched gene and evidence-level analysis exploratory because it was specified after the pre-registered analysis, which yielded null-to-weak results. The study tested judgments on curated oncology knowledge bases; the supplied abstract does not report patient treatment outcomes. The correct-evidence test used an oracle condition rather than retrieval, so it does not assess whether models can retrieve the appropriate evidence.
Open question
How would the models’ resistance judgments change when evidence is weaker or comes from different reference knowledge bases?
Source-stated
The study evaluated ten models on 1,464 resistance cases and found therapy endorsements in 21.8–81.4% of cases.
We evaluated ten models on 1,464 resistance cases from five curated oncology knowledge bases, collecting 58,910 judgments. Models endorsed resisted therapies in 21.8-81.4% of cases.
Source-stated
An endorsement rate alone may reflect agreement tendency rather than distinguishing resistance from sensitivity: one open-weight model endorsed 81.4% of resistance cases and 82.3% of sensitivity controls.
one open-weight model endorsed 81.4% of resistance cases and 82.3% of sensitivity controls.
Source-stated
The authors describe matched gene and evidence-level analysis as exploratory, and report that measured failures depended on the reference knowledge base and evidence strength.
This analysis is exploratory because it was specified after the pre-registered analysis, which yielded null-to-weak results. Measured failures depended strongly on the reference knowledge base and evidence strength.
Source-stated
When a second source independently confirmed resistance, four models across three vendors endorsed at most 2.4% of cases; at weaker evidence tiers, discrimination was not measurable.
When a second source independently confirmed resistance, four models across three vendors endorsed at most 2.4% of cases; at weaker evidence tiers, discrimination was not measurable.
Source-stated
The authors report that repeating a model’s own resistance conclusion reduced endorsement, while falsely attributing the opposite conclusion increased endorsement.
Repeating a model's own resistance conclusion reduced endorsement to 0.0-3.5% on cases where it had stated resistance. Falsely attributing the opposite conclusion increased endorsement above baseline for all seven hosted models
Source-stated
Correct curated evidence reduced unqualified strong recommendations in an oracle condition, which tests evidence use rather than retrieval.
Providing the correct curated evidence reduced unqualified strong recommendations to 0.3-3.5%, testing evidence use under an oracle condition rather than retrieval.
Source and contributor context
Ashrafi, Negin, Daneshjou, Roxana · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Open weights
A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 2a608f40-2f33-419f-8090-de9825f2fbe6.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-17T10:32:01.073Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: a6d63939c43806924c1195a9978d6418c85a2136e30342d90bc587a1e9cef2e4. Source text hash: 0731287b038d1a279a3d4a545d450f9ff8a7ee33445e7ae9c6b23277c1c47b60.
Revision and source-check history
Version 1 · 2026-11-17T10:32:01.073Z
Researchers characterize when clinical LLMs recommend therapies despite genomic resistance
Negin Ashrafi and Roxana Daneshjou report that, across ten models tested on 1,464 resistance cases, models endorsed therapies despite genomic evidence of resistance in 21.8–81.4% of cases. But an endorsement rate alone can confuse resistance handling with a general tendency to agree: one endorsed nearly as many sensitivity controls as resistance cases. The authors’ exploratory matched-group analysis found that results depended strongly on the reference knowledge base and evidence strength. When a second source independently confirmed resistance, four models across three vendors endorsed at most 2.4% of cases; at weaker evidence tiers, discrimination was not measurable. The paper also reports that models sometimes acknowledged resistance and still recommended the therapy. Repeating a model’s own resistance conclusion reduced endorsements, while falsely attributing the opposite conclusion raised them; providing correct curated evidence reduced unqualified strong recommendations. That last test used an oracle condition, not retrieval, so it does not establish how well models find the right evidence in practice. The findings describe judgments on curated cases, not treatment outcomes for patients.
Original source
cs.AI · Article
Source details
Original source date: Nov 14, 2026 (2026-11-14).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-17T10:32:01.073Z. First observed: 2026-11-17T08:31:07.176Z. Retrieved: 2026-11-17T10:19:34.068Z.
Limits
The authors call the matched gene and evidence-level analysis exploratory because it was specified after the pre-registered analysis, which yielded null-to-weak results. The study tested judgments on curated oncology knowledge bases; the supplied abstract does not report patient treatment outcomes. The correct-evidence test used an oracle condition rather than retrieval, so it does not assess whether models can retrieve the appropriate evidence.
Open question
How would the models’ resistance judgments change when evidence is weaker or comes from different reference knowledge bases?
Source-stated
The study evaluated ten models on 1,464 resistance cases and found therapy endorsements in 21.8–81.4% of cases.
We evaluated ten models on 1,464 resistance cases from five curated oncology knowledge bases, collecting 58,910 judgments. Models endorsed resisted therapies in 21.8-81.4% of cases.
Source-stated
An endorsement rate alone may reflect agreement tendency rather than distinguishing resistance from sensitivity: one open-weight model endorsed 81.4% of resistance cases and 82.3% of sensitivity controls.
one open-weight model endorsed 81.4% of resistance cases and 82.3% of sensitivity controls.
Source-stated
The authors describe matched gene and evidence-level analysis as exploratory, and report that measured failures depended on the reference knowledge base and evidence strength.
This analysis is exploratory because it was specified after the pre-registered analysis, which yielded null-to-weak results. Measured failures depended strongly on the reference knowledge base and evidence strength.
Source-stated
When a second source independently confirmed resistance, four models across three vendors endorsed at most 2.4% of cases; at weaker evidence tiers, discrimination was not measurable.
When a second source independently confirmed resistance, four models across three vendors endorsed at most 2.4% of cases; at weaker evidence tiers, discrimination was not measurable.
Source-stated
The authors report that repeating a model’s own resistance conclusion reduced endorsement, while falsely attributing the opposite conclusion increased endorsement.
Repeating a model's own resistance conclusion reduced endorsement to 0.0-3.5% on cases where it had stated resistance. Falsely attributing the opposite conclusion increased endorsement above baseline for all seven hosted models
Source-stated
Correct curated evidence reduced unqualified strong recommendations in an oracle condition, which tests evidence use rather than retrieval.
Providing the correct curated evidence reduced unqualified strong recommendations to 0.3-3.5%, testing evidence use under an oracle condition rather than retrieval.
Source and contributor context
Ashrafi, Negin, Daneshjou, Roxana · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Open weights
A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 2a608f40-2f33-419f-8090-de9825f2fbe6.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-17T10:32:01.073Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: a6d63939c43806924c1195a9978d6418c85a2136e30342d90bc587a1e9cef2e4. Source text hash: 0731287b038d1a279a3d4a545d450f9ff8a7ee33445e7ae9c6b23277c1c47b60.
uncertain · 2026-11-17T10:32:01.073Z · anthropic/claude-opus-5.5
Publication: 2a608f40-2f33-419f-8090-de9825f2fbe6
attribution — pass: The draft correctly attributes the study to Negin Ashrafi and Roxana Daneshjou, and its findings are framed as what the paper reports rather than as established fact.
Authors:Negin Ashrafi, Roxana Daneshjou
summary — uncertain: The summary omits why this matters to a general reader: the abstract frames the study around language models being proposed to support molecular tumour boards. Fix: add one sentence on that use case. Central terms such as sensitivity controls, discrimination and oracle condition are not defined; tooltip entries would fix this. 'Nearly as many sensitivity controls' slightly inverts the figures (82.3% of controls vs 81.4% of resistance cases); 'about as often' would be accurate. The false-attribution result loses force: the draft drops 95.2% for GPT-4o and the 0.9-point no-attribution control, which isolate the attributed-conclusion effect. The question is a fair open issue.
Language models are increasingly proposed to support molecular tumour boards, but may recommend cancer therapies despite genomic evidence of resistance.
date — pass: originalPublished is 2026-11-14, which matches the v1 submission date on arXiv. The later feed date is correctly not used.
[Submitted on 14 Nov 2026]
support — pass: Every claim is backed by a verbatim abstract excerpt. The caveats are grounded in the source: the exploratory analysis came after a pre-registered one with null-to-weak results, and the evidence test used an oracle condition rather than retrieval. The note that the findings are judgments on curated cases rather than patient outcomes is a conservative limitation, not an invented claim. No unsupported benefits or costs are implied.
testing evidence use under an oracle condition rather than retrieval.
media — pass: The selected media item is the verified arXiv abstract page for this paper, which is the direct original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv