# Liu and Yang test whether an LLM can audit clinical prediction models through queries alone

[Canonical reading](https://escapemeridian.com/readings/liu-and-yang-test-whether-an-llm-can-audit-clinical-prediction-d6aab00c)

Zhiyu Liu and Xinyu Yang report a blinded test of whether GPT-6 Astra could explain and audit black-box clinical prediction models using only their predictions. They trained four kinds of model on data from the International Stroke Trial, made 24 flawed copies, and included 8 clean models. Across three audits of each model, GPT-6 Astra found the planted flaw in 69 of 72 runs; a purpose-written script found all 24 flaws. The LLM also produced surrogate models that closely reproduced the audited models’ log-odds, a measure of their predictions. But its reports of problems were not dependable without checking: 60% of 1,089 checkable reports failed a pre-specified check against average model behaviour. The authors say most of those reports described real but small steps, while 15% of checkable reports were not found in the model. Of 601 unexpected relationships reported from clean models, 11% held in patients randomised later. The authors’ conclusion is qualified: an LLM with query access alone can describe a black-box model and find nearly every planted flaw in this test, but reports of problems must be checked against the model and discoveries against data. How well this approach performs on clinical prediction models beyond the models tested remains unresolved.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

## Original source

arXiv cs.AI · Article

Original source date: Nov 18, 2026 (2026-11-18).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-19T09:17:42.604Z. First observed: 2026-11-19T05:01:38.287Z. Retrieved: 2026-11-19T09:16:38.686Z.

- [Auditing black-box clinical prediction models with a large language model through queries alone: a blinded test with planted flaws](https://arxiv.org/abs/2611.24877)

## Limits

The experiment used models trained on data from the International Stroke Trial and planted flaws in copies; these results do not establish performance on clinical prediction models beyond the test. The abstract reports that most failed checks described real but small steps, but does not give further detail about those steps or how their significance was judged.


## Open question

How well would query-only LLM auditing perform on clinical prediction models beyond those tested in this study?


## Source-stated

The study tested GPT-6 Astra auditing models through a tool that returned only their predictions, using blinded runs and precommitted judging rules.

> GPT-6 Astra audited each model three times, blind, through a tool that only returned the model's predictions; the judging rules were committed before any run

- [Supporting source](https://arxiv.org/abs/2611.24877)

## Source-stated

The experiment used four kinds of model trained on International Stroke Trial data, with one planted flaw in each of 24 model copies and 8 clean models.

> we trained four kinds of model (logistic regression, explainable boosting machine, gradient-boosted trees, neural network) for two outcomes on patients randomised earlier, planted one flaw in each of 24 copies

- [Supporting source](https://arxiv.org/abs/2611.24877)

## Source-stated

GPT-6 Astra found planted flaws in 69 of 72 audit runs; a purpose-written script found all 24 planted flaws.

> It found the planted flaw in 69 of 72 runs (96%; 95% CI 80% to 99%); a script written for the purpose found all 24.

- [Supporting source](https://arxiv.org/abs/2611.24877)

## Source-stated

Many reports of problems did not pass the pre-specified check against the models’ average behaviour, and the authors say reports require checking against the model.

> 60% of 1,089 checkable reports failed the pre-specified check against the model's average behaviour; post hoc, most of these were real but small steps

- [Supporting source](https://arxiv.org/abs/2611.24877)

## Source-stated

Only 11% of 601 unexpected relationships reported from clean models held in patients randomised later.

> Of 601 unexpected relationships it read out of the clean models, 11% held in patients randomised later

- [Supporting source](https://arxiv.org/abs/2611.24877)

## Escape Meridian interpretation

The findings establish performance in this particular test, not how the approach performs on other clinical prediction models.

> With query access alone, an LLM can describe a black-box model faithfully and find nearly every planted flaw

- [Supporting source](https://arxiv.org/abs/2611.24877)

## Source and contributor context

Liu, Zhiyu, Yang, Xinyu · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Large language model

A language model with many learned numerical settings, used to process and generate language.

- [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: 8f16308d-bce7-447f-8a94-a9c91b554b6f.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-19T09:17:42.604Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 549b21e3ab255d4cd6864ed386c1dd24606dcb37920f30652e39166f96daaaf9. Source text hash: f65d34a7fc42cbeaf2f7e50dea62871d37ac714d7c1ef5db4acf8dfdd5254da3.


## Revision and source-check history

### Version 1 — 2026-11-19T09:17:42.604Z

Zhiyu Liu and Xinyu Yang report a blinded test of whether GPT-6 Astra could explain and audit black-box clinical prediction models using only their predictions. They trained four kinds of model on data from the International Stroke Trial, made 24 flawed copies, and included 8 clean models. Across three audits of each model, GPT-6 Astra found the planted flaw in 69 of 72 runs; a purpose-written script found all 24 flaws. The LLM also produced surrogate models that closely reproduced the audited models’ log-odds, a measure of their predictions. But its reports of problems were not dependable without checking: 60% of 1,089 checkable reports failed a pre-specified check against average model behaviour. The authors say most of those reports described real but small steps, while 15% of checkable reports were not found in the model. Of 601 unexpected relationships reported from clean models, 11% held in patients randomised later. The authors’ conclusion is qualified: an LLM with query access alone can describe a black-box model and find nearly every planted flaw in this test, but reports of problems must be checked against the model and discoveries against data. How well this approach performs on clinical prediction models beyond the models tested remains unresolved.

## Original source

arXiv cs.AI · Article

Original source date: Nov 18, 2026 (2026-11-18).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-19T09:17:42.604Z. First observed: 2026-11-19T05:01:38.287Z. Retrieved: 2026-11-19T09:16:38.686Z.

- [Auditing black-box clinical prediction models with a large language model through queries alone: a blinded test with planted flaws](https://arxiv.org/abs/2611.24877)

## Limits

The experiment used models trained on data from the International Stroke Trial and planted flaws in copies; these results do not establish performance on clinical prediction models beyond the test. The abstract reports that most failed checks described real but small steps, but does not give further detail about those steps or how their significance was judged.


## Open question

How well would query-only LLM auditing perform on clinical prediction models beyond those tested in this study?


## Source-stated

The study tested GPT-6 Astra auditing models through a tool that returned only their predictions, using blinded runs and precommitted judging rules.

> GPT-6 Astra audited each model three times, blind, through a tool that only returned the model's predictions; the judging rules were committed before any run

- [Supporting source](https://arxiv.org/abs/2611.24877)

## Source-stated

The experiment used four kinds of model trained on International Stroke Trial data, with one planted flaw in each of 24 model copies and 8 clean models.

> we trained four kinds of model (logistic regression, explainable boosting machine, gradient-boosted trees, neural network) for two outcomes on patients randomised earlier, planted one flaw in each of 24 copies

- [Supporting source](https://arxiv.org/abs/2611.24877)

## Source-stated

GPT-6 Astra found planted flaws in 69 of 72 audit runs; a purpose-written script found all 24 planted flaws.

> It found the planted flaw in 69 of 72 runs (96%; 95% CI 80% to 99%); a script written for the purpose found all 24.

- [Supporting source](https://arxiv.org/abs/2611.24877)

## Source-stated

Many reports of problems did not pass the pre-specified check against the models’ average behaviour, and the authors say reports require checking against the model.

> 60% of 1,089 checkable reports failed the pre-specified check against the model's average behaviour; post hoc, most of these were real but small steps

- [Supporting source](https://arxiv.org/abs/2611.24877)

## Source-stated

Only 11% of 601 unexpected relationships reported from clean models held in patients randomised later.

> Of 601 unexpected relationships it read out of the clean models, 11% held in patients randomised later

- [Supporting source](https://arxiv.org/abs/2611.24877)

## Escape Meridian interpretation

The findings establish performance in this particular test, not how the approach performs on other clinical prediction models.

> With query access alone, an LLM can describe a black-box model faithfully and find nearly every planted flaw

- [Supporting source](https://arxiv.org/abs/2611.24877)

## Source and contributor context

Liu, Zhiyu, Yang, Xinyu · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Large language model

A language model with many learned numerical settings, used to process and generate language.

- [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: 8f16308d-bce7-447f-8a94-a9c91b554b6f.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-19T09:17:42.604Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 549b21e3ab255d4cd6864ed386c1dd24606dcb37920f30652e39166f96daaaf9. Source text hash: f65d34a7fc42cbeaf2f7e50dea62871d37ac714d7c1ef5db4acf8dfdd5254da3.


### uncertain — 2026-11-19T09:17:42.604Z — anthropic/claude-opus-5.5

Publication: 8f16308d-bce7-447f-8a94-a9c91b554b6f

attribution — pass: The reading correctly attributes the study to Zhiyu Liu and Xinyu Yang and keeps the results and conclusion as the authors' reported findings.

> Authors:Zhiyu Liu, Xinyu Yang

summary — uncertain: The reading is mostly faithful, but it gives the 11% discovery rate without the abstract's comparators: random pairs 6%, a regression scan 28%, and 33% vs 30% after a training-data check. Without them, readers cannot judge the result; the smallest fix is to add the baselines. It drops 'post hoc' from the small-steps finding and the word 'faithfully' from the authors' conclusion. It also places 'in this test' inside the authors' conclusion, which the abstract does not do. Central terms such as surrogate model, black-box and 'randomised later' are not explained for non-specialists.

> 11% held in patients randomised later (random pairs 6%; a regression scan's shorter list of its strongest candidates, 28%); after a check on the training data, 33% against 30%

date — pass: The originalPublished date of 2026-11-18 matches the v1 submission date on arXiv.

> \[Submitted on 18 Nov 2026\]

support — uncertain: The source-stated claims match the abstract. The claim 'Only 11% of 601 ... held' adds evaluative framing and omits the 6% random baseline, which the LLM exceeded, so 'Only' is not supported as stated. The interpretation claim about scope is reasonably conditional, but its excerpt is the authors' positive conclusion, which does not itself support the limitation. Dropping 'Only' or adding the comparators would repair the claim.

> Of 601 unexpected relationships it read out of the clean models, 11% held in patients randomised later (random pairs 6%

media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most relevant way to access the source.

## Recorded references

Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter.

No public references recorded here yet.

[Reference data (JSON)](/readings/d6aab00c76f20543b21c6c2c2e4b191ee8913936045c076b78025e9daff32e5a/references.json)

## Corrections and challenges

[Send a correction or an opposing source](https://escapemeridian.com/corrections/)

[JSON record](https://escapemeridian.com/readings/liu-and-yang-test-whether-an-llm-can-audit-clinical-prediction-d6aab00c.json)
