{"id":"d6aab00c76f20543b21c6c2c2e4b191ee8913936045c076b78025e9daff32e5a","title":"Liu and Yang test whether an LLM can audit clinical prediction models through queries alone","argument":"Zhiyu Liu and Xinyu Yang report a blinded test of whether GPT-6 Astra could explain and audit black-box clinical prediction models using only their predictions. They trained four kinds of model on data from the International Stroke Trial, made 24 flawed copies, and included 8 clean models. Across three audits of each model, GPT-6 Astra found the planted flaw in 69 of 72 runs; a purpose-written script found all 24 flaws. The LLM also produced surrogate models that closely reproduced the audited models’ log-odds, a measure of their predictions. But its reports of problems were not dependable without checking: 60% of 1,089 checkable reports failed a pre-specified check against average model behaviour. The authors say most of those reports described real but small steps, while 15% of checkable reports were not found in the model. Of 601 unexpected relationships reported from clean models, 11% held in patients randomised later. The authors’ conclusion is qualified: an LLM with query access alone can describe a black-box model and find nearly every planted flaw in this test, but reports of problems must be checked against the model and discoveries against data. How well this approach performs on clinical prediction models beyond the models tested remains unresolved.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The experiment used models trained on data from the International Stroke Trial and planted flaws in copies; these results do not establish performance on clinical prediction models beyond the test. The abstract reports that most failed checks described real but small steps, but does not give further detail about those steps or how their significance was judged.","speaker":{"name":"Liu, Zhiyu, Yang, Xinyu","roleAtSource":"Article author"},"source":{"title":"Auditing black-box clinical prediction models with a large language model through queries alone: a blinded test with planted flaws","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.24877","publishedAt":"2026-11-18","dateBasis":"Date basis not provided."},"evidence":[{"text":"The study tested GPT-6 Astra auditing models through a tool that returned only their predictions, using blinded runs and precommitted judging rules.","excerpt":"GPT-6 Astra audited each model three times, blind, through a tool that only returned the model's predictions; the judging rules were committed before any run","url":"https://arxiv.org/abs/2611.24877","relationship":"Source-stated"},{"text":"The experiment used four kinds of model trained on International Stroke Trial data, with one planted flaw in each of 24 model copies and 8 clean models.","excerpt":"we trained four kinds of model (logistic regression, explainable boosting machine, gradient-boosted trees, neural network) for two outcomes on patients randomised earlier, planted one flaw in each of 24 copies","url":"https://arxiv.org/abs/2611.24877","relationship":"Source-stated"},{"text":"GPT-6 Astra found planted flaws in 69 of 72 audit runs; a purpose-written script found all 24 planted flaws.","excerpt":"It found the planted flaw in 69 of 72 runs (96%; 95% CI 80% to 99%); a script written for the purpose found all 24.","url":"https://arxiv.org/abs/2611.24877","relationship":"Source-stated"},{"text":"Many reports of problems did not pass the pre-specified check against the models’ average behaviour, and the authors say reports require checking against the model.","excerpt":"60% of 1,089 checkable reports failed the pre-specified check against the model's average behaviour; post hoc, most of these were real but small steps","url":"https://arxiv.org/abs/2611.24877","relationship":"Source-stated"},{"text":"Only 11% of 601 unexpected relationships reported from clean models held in patients randomised later.","excerpt":"Of 601 unexpected relationships it read out of the clean models, 11% held in patients randomised later","url":"https://arxiv.org/abs/2611.24877","relationship":"Source-stated"},{"text":"The findings establish performance in this particular test, not how the approach performs on other clinical prediction models.","excerpt":"With query access alone, an LLM can describe a black-box model faithfully and find nearly every planted flaw","url":"https://arxiv.org/abs/2611.24877","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.24877","checkedAt":"2026-11-19T09:48:10.304Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well would query-only LLM auditing perform on clinical prediction models beyond those tested in this study?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Liu and Yang test whether an LLM can audit clinical prediction models through queries alone","summary":"Zhiyu Liu and Xinyu Yang report a blinded test of whether GPT-6 Astra could explain and audit black-box clinical prediction models using only their predictions. They trained four kinds of model on data from the International Stroke Trial, made 24 flawed copies, and included 8 clean models. Across three audits of each model, GPT-6 Astra found the planted flaw in 69 of 72 runs; a purpose-written script found all 24 flaws. The LLM also produced surrogate models that closely reproduced the audited models’ log-odds, a measure of their predictions. But its reports of problems were not dependable without checking: 60% of 1,089 checkable reports failed a pre-specified check against average model behaviour. The authors say most of those reports described real but small steps, while 15% of checkable reports were not found in the model. Of 601 unexpected relationships reported from clean models, 11% held in patients randomised later. The authors’ conclusion is qualified: an LLM with query access alone can describe a black-box model and find nearly every planted flaw in this test, but reports of problems must be checked against the model and discoveries against data. How well this approach performs on clinical prediction models beyond the models tested remains unresolved.","question":"How well would query-only LLM auditing perform on clinical prediction models beyond those tested in this study?","claims":[{"text":"The study tested GPT-6 Astra auditing models through a tool that returned only their predictions, using blinded runs and precommitted judging rules.","excerpt":"GPT-6 Astra audited each model three times, blind, through a tool that only returned the model's predictions; the judging rules were committed before any run","support":"source-stated"},{"text":"The experiment used four kinds of model trained on International Stroke Trial data, with one planted flaw in each of 24 model copies and 8 clean models.","excerpt":"we trained four kinds of model (logistic regression, explainable boosting machine, gradient-boosted trees, neural network) for two outcomes on patients randomised earlier, planted one flaw in each of 24 copies","support":"source-stated"},{"text":"GPT-6 Astra found planted flaws in 69 of 72 audit runs; a purpose-written script found all 24 planted flaws.","excerpt":"It found the planted flaw in 69 of 72 runs (96%; 95% CI 80% to 99%); a script written for the purpose found all 24.","support":"source-stated"},{"text":"Many reports of problems did not pass the pre-specified check against the models’ average behaviour, and the authors say reports require checking against the model.","excerpt":"60% of 1,089 checkable reports failed the pre-specified check against the model's average behaviour; post hoc, most of these were real but small steps","support":"source-stated"},{"text":"Only 11% of 601 unexpected relationships reported from clean models held in patients randomised later.","excerpt":"Of 601 unexpected relationships it read out of the clean models, 11% held in patients randomised later","support":"source-stated"},{"text":"The findings establish performance in this particular test, not how the approach performs on other clinical prediction models.","excerpt":"With query access alone, an LLM can describe a black-box model faithfully and find nearly every planted flaw","support":"interpretation"}],"limitations":["The experiment used models trained on data from the International Stroke Trial and planted flaws in copies; these results do not establish performance on clinical prediction models beyond the test.","The abstract reports that most failed checks described real but small steps, but does not give further detail about those steps or how their significance was judged."],"mediaId":"d12c82b1c69de90c046175ac59c04011603bb2e21483f62caafa55999926a010","correctionReason":null,"id":"d6aab00c76f20543b21c6c2c2e4b191ee8913936045c076b78025e9daff32e5a","publicationId":"8f16308d-bce7-447f-8a94-a9c91b554b6f","version":1,"sourceUrl":"https://arxiv.org/abs/2611.24877","sourceTitle":"Auditing black-box clinical prediction models with a large language model through queries alone: a blinded test with planted flaws","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-18","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-19T05:01:38.287Z","createdAt":"2026-11-19T09:17:42.604Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"d12c82b1c69de90c046175ac59c04011603bb2e21483f62caafa55999926a010","kind":"article","url":"https://arxiv.org/abs/2611.24877","sourceUrl":"https://arxiv.org/abs/2611.24877","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-19T09:16:38.950Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.24877","checkedAt":"2026-11-19T09:48:10.304Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"d12c82b1c69de90c046175ac59c04011603bb2e21483f62caafa55999926a010","confidence":0.7,"provider":"typesafe","model":"jev-1.13.0","inputHash":"f403b5201500b54a7750403c050c31f25f299b4ee378e464ac07b3f8f1d64c9f","decidedAt":"2026-11-19T09:16:40.557Z"},"provenance":{"textHash":"f65d34a7fc42cbeaf2f7e50dea62871d37ac714d7c1ef5db4acf8dfdd5254da3","observationId":"549b21e3ab255d4cd6864ed386c1dd24606dcb37920f30652e39166f96daaaf9","retrievedAt":"2026-11-19T09:16:38.686Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"e7e2443a-6be3-474c-879a-562927587e3c"},"reviewer":{"model":"gpt-6-luna","runId":"992ab083-a444-4afe-8e24-9eddf47c77ee"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-19T09:17:42.604Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Liu, Zhiyu","url":null,"kind":"unknown"},{"name":"Yang, Xinyu","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Liu, Zhiyu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Liu, Zhiyu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24877","locator":"head/meta[name=author|citation_author]"},"assertionId":"ca0261966964b278f3f5218a7ce1f8dc99b5fdde2f2a7d0b9df91fef3f5947b5:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T09:48:10.304Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ca0261966964b278f3f5218a7ce1f8dc99b5fdde2f2a7d0b9df91fef3f5947b5","sourceUrl":"https://arxiv.org/abs/2611.24877","contentSha256":"56d4d721fa629f80394a3effe505ecb0c0ba3aaf8934b2f76b8054dff9cd92a2","textHash":"f65d34a7fc42cbeaf2f7e50dea62871d37ac714d7c1ef5db4acf8dfdd5254da3","observedAt":"2026-11-19T09:48:10.304Z"}},{"actor":{"name":"Yang, Xinyu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yang, Xinyu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24877","locator":"head/meta[name=author|citation_author]"},"assertionId":"ca0261966964b278f3f5218a7ce1f8dc99b5fdde2f2a7d0b9df91fef3f5947b5:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T09:48:10.304Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ca0261966964b278f3f5218a7ce1f8dc99b5fdde2f2a7d0b9df91fef3f5947b5","sourceUrl":"https://arxiv.org/abs/2611.24877","contentSha256":"56d4d721fa629f80394a3effe505ecb0c0ba3aaf8934b2f76b8054dff9cd92a2","textHash":"f65d34a7fc42cbeaf2f7e50dea62871d37ac714d7c1ef5db4acf8dfdd5254da3","observedAt":"2026-11-19T09:48:10.304Z"}}],"participationCoverage":{"total":2,"returned":2,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.24877","snapshotId":"ca0261966964b278f3f5218a7ce1f8dc99b5fdde2f2a7d0b9df91fef3f5947b5","contentSha256":"56d4d721fa629f80394a3effe505ecb0c0ba3aaf8934b2f76b8054dff9cd92a2","textHash":"f65d34a7fc42cbeaf2f7e50dea62871d37ac714d7c1ef5db4acf8dfdd5254da3","observedAt":"2026-11-19T09:48:10.304Z","basis":"publisher-page-metadata","sourceLinks":42,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/d6aab00c76f20543b21c6c2c2e4b191ee8913936045c076b78025e9daff32e5a/source"}},"canonicalUrl":"https://escapemeridian.com/readings/liu-and-yang-test-whether-an-llm-can-audit-clinical-prediction-d6aab00c","formats":{"html":"/readings/liu-and-yang-test-whether-an-llm-can-audit-clinical-prediction-d6aab00c","markdown":"/readings/liu-and-yang-test-whether-an-llm-can-audit-clinical-prediction-d6aab00c.md","json":"/readings/liu-and-yang-test-whether-an-llm-can-audit-clinical-prediction-d6aab00c.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"d6aab00c76f20543b21c6c2c2e4b191ee8913936045c076b78025e9daff32e5a","revision":"8f16308d-bce7-447f-8a94-a9c91b554b6f","revisionKind":"publication-id","title":"Liu and Yang test whether an LLM can audit clinical prediction models through queries alone","url":"https://escapemeridian.com/readings/liu-and-yang-test-whether-an-llm-can-audit-clinical-prediction-d6aab00c"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.24877","revision":"f65d34a7fc42cbeaf2f7e50dea62871d37ac714d7c1ef5db4acf8dfdd5254da3","revisionKind":"sha256-source-text","title":"Auditing black-box clinical prediction models with a large language model through queries alone: a blinded test with planted flaws","url":"https://arxiv.org/abs/2611.24877"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter."},"revisions":[{"id":"d6aab00c76f20543b21c6c2c2e4b191ee8913936045c076b78025e9daff32e5a","title":"Liu and Yang test whether an LLM can audit clinical prediction models through queries alone","argument":"Zhiyu Liu and Xinyu Yang report a blinded test of whether GPT-6 Astra could explain and audit black-box clinical prediction models using only their predictions. They trained four kinds of model on data from the International Stroke Trial, made 24 flawed copies, and included 8 clean models. Across three audits of each model, GPT-6 Astra found the planted flaw in 69 of 72 runs; a purpose-written script found all 24 flaws. The LLM also produced surrogate models that closely reproduced the audited models’ log-odds, a measure of their predictions. But its reports of problems were not dependable without checking: 60% of 1,089 checkable reports failed a pre-specified check against average model behaviour. The authors say most of those reports described real but small steps, while 15% of checkable reports were not found in the model. Of 601 unexpected relationships reported from clean models, 11% held in patients randomised later. The authors’ conclusion is qualified: an LLM with query access alone can describe a black-box model and find nearly every planted flaw in this test, but reports of problems must be checked against the model and discoveries against data. How well this approach performs on clinical prediction models beyond the models tested remains unresolved.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The experiment used models trained on data from the International Stroke Trial and planted flaws in copies; these results do not establish performance on clinical prediction models beyond the test. The abstract reports that most failed checks described real but small steps, but does not give further detail about those steps or how their significance was judged.","speaker":{"name":"Liu, Zhiyu, Yang, Xinyu","roleAtSource":"Article author"},"source":{"title":"Auditing black-box clinical prediction models with a large language model through queries alone: a blinded test with planted flaws","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.24877","publishedAt":"2026-11-18","dateBasis":"Date basis not provided."},"evidence":[{"text":"The study tested GPT-6 Astra auditing models through a tool that returned only their predictions, using blinded runs and precommitted judging rules.","excerpt":"GPT-6 Astra audited each model three times, blind, through a tool that only returned the model's predictions; the judging rules were committed before any run","url":"https://arxiv.org/abs/2611.24877","relationship":"Source-stated"},{"text":"The experiment used four kinds of model trained on International Stroke Trial data, with one planted flaw in each of 24 model copies and 8 clean models.","excerpt":"we trained four kinds of model (logistic regression, explainable boosting machine, gradient-boosted trees, neural network) for two outcomes on patients randomised earlier, planted one flaw in each of 24 copies","url":"https://arxiv.org/abs/2611.24877","relationship":"Source-stated"},{"text":"GPT-6 Astra found planted flaws in 69 of 72 audit runs; a purpose-written script found all 24 planted flaws.","excerpt":"It found the planted flaw in 69 of 72 runs (96%; 95% CI 80% to 99%); a script written for the purpose found all 24.","url":"https://arxiv.org/abs/2611.24877","relationship":"Source-stated"},{"text":"Many reports of problems did not pass the pre-specified check against the models’ average behaviour, and the authors say reports require checking against the model.","excerpt":"60% of 1,089 checkable reports failed the pre-specified check against the model's average behaviour; post hoc, most of these were real but small steps","url":"https://arxiv.org/abs/2611.24877","relationship":"Source-stated"},{"text":"Only 11% of 601 unexpected relationships reported from clean models held in patients randomised later.","excerpt":"Of 601 unexpected relationships it read out of the clean models, 11% held in patients randomised later","url":"https://arxiv.org/abs/2611.24877","relationship":"Source-stated"},{"text":"The findings establish performance in this particular test, not how the approach performs on other clinical prediction models.","excerpt":"With query access alone, an LLM can describe a black-box model faithfully and find nearly every planted flaw","url":"https://arxiv.org/abs/2611.24877","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.24877","checkedAt":"2026-11-19T09:48:10.304Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well would query-only LLM auditing perform on clinical prediction models beyond those tested in this study?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Liu and Yang test whether an LLM can audit clinical prediction models through queries alone","summary":"Zhiyu Liu and Xinyu Yang report a blinded test of whether GPT-6 Astra could explain and audit black-box clinical prediction models using only their predictions. They trained four kinds of model on data from the International Stroke Trial, made 24 flawed copies, and included 8 clean models. Across three audits of each model, GPT-6 Astra found the planted flaw in 69 of 72 runs; a purpose-written script found all 24 flaws. The LLM also produced surrogate models that closely reproduced the audited models’ log-odds, a measure of their predictions. But its reports of problems were not dependable without checking: 60% of 1,089 checkable reports failed a pre-specified check against average model behaviour. The authors say most of those reports described real but small steps, while 15% of checkable reports were not found in the model. Of 601 unexpected relationships reported from clean models, 11% held in patients randomised later. The authors’ conclusion is qualified: an LLM with query access alone can describe a black-box model and find nearly every planted flaw in this test, but reports of problems must be checked against the model and discoveries against data. How well this approach performs on clinical prediction models beyond the models tested remains unresolved.","question":"How well would query-only LLM auditing perform on clinical prediction models beyond those tested in this study?","claims":[{"text":"The study tested GPT-6 Astra auditing models through a tool that returned only their predictions, using blinded runs and precommitted judging rules.","excerpt":"GPT-6 Astra audited each model three times, blind, through a tool that only returned the model's predictions; the judging rules were committed before any run","support":"source-stated"},{"text":"The experiment used four kinds of model trained on International Stroke Trial data, with one planted flaw in each of 24 model copies and 8 clean models.","excerpt":"we trained four kinds of model (logistic regression, explainable boosting machine, gradient-boosted trees, neural network) for two outcomes on patients randomised earlier, planted one flaw in each of 24 copies","support":"source-stated"},{"text":"GPT-6 Astra found planted flaws in 69 of 72 audit runs; a purpose-written script found all 24 planted flaws.","excerpt":"It found the planted flaw in 69 of 72 runs (96%; 95% CI 80% to 99%); a script written for the purpose found all 24.","support":"source-stated"},{"text":"Many reports of problems did not pass the pre-specified check against the models’ average behaviour, and the authors say reports require checking against the model.","excerpt":"60% of 1,089 checkable reports failed the pre-specified check against the model's average behaviour; post hoc, most of these were real but small steps","support":"source-stated"},{"text":"Only 11% of 601 unexpected relationships reported from clean models held in patients randomised later.","excerpt":"Of 601 unexpected relationships it read out of the clean models, 11% held in patients randomised later","support":"source-stated"},{"text":"The findings establish performance in this particular test, not how the approach performs on other clinical prediction models.","excerpt":"With query access alone, an LLM can describe a black-box model faithfully and find nearly every planted flaw","support":"interpretation"}],"limitations":["The experiment used models trained on data from the International Stroke Trial and planted flaws in copies; these results do not establish performance on clinical prediction models beyond the test.","The abstract reports that most failed checks described real but small steps, but does not give further detail about those steps or how their significance was judged."],"mediaId":"d12c82b1c69de90c046175ac59c04011603bb2e21483f62caafa55999926a010","correctionReason":null,"id":"d6aab00c76f20543b21c6c2c2e4b191ee8913936045c076b78025e9daff32e5a","publicationId":"8f16308d-bce7-447f-8a94-a9c91b554b6f","version":1,"sourceUrl":"https://arxiv.org/abs/2611.24877","sourceTitle":"Auditing black-box clinical prediction models with a large language model through queries alone: a blinded test with planted flaws","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-18","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-19T05:01:38.287Z","createdAt":"2026-11-19T09:17:42.604Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"d12c82b1c69de90c046175ac59c04011603bb2e21483f62caafa55999926a010","kind":"article","url":"https://arxiv.org/abs/2611.24877","sourceUrl":"https://arxiv.org/abs/2611.24877","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-19T09:16:38.950Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.24877","checkedAt":"2026-11-19T09:48:10.304Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"d12c82b1c69de90c046175ac59c04011603bb2e21483f62caafa55999926a010","confidence":0.7,"provider":"typesafe","model":"jev-1.13.0","inputHash":"f403b5201500b54a7750403c050c31f25f299b4ee378e464ac07b3f8f1d64c9f","decidedAt":"2026-11-19T09:16:40.557Z"},"provenance":{"textHash":"f65d34a7fc42cbeaf2f7e50dea62871d37ac714d7c1ef5db4acf8dfdd5254da3","observationId":"549b21e3ab255d4cd6864ed386c1dd24606dcb37920f30652e39166f96daaaf9","retrievedAt":"2026-11-19T09:16:38.686Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"e7e2443a-6be3-474c-879a-562927587e3c"},"reviewer":{"model":"gpt-6-luna","runId":"992ab083-a444-4afe-8e24-9eddf47c77ee"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-19T09:17:42.604Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Liu, Zhiyu","url":null,"kind":"unknown"},{"name":"Yang, Xinyu","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Liu, Zhiyu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Liu, Zhiyu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24877","locator":"head/meta[name=author|citation_author]"},"assertionId":"ca0261966964b278f3f5218a7ce1f8dc99b5fdde2f2a7d0b9df91fef3f5947b5:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T09:48:10.304Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ca0261966964b278f3f5218a7ce1f8dc99b5fdde2f2a7d0b9df91fef3f5947b5","sourceUrl":"https://arxiv.org/abs/2611.24877","contentSha256":"56d4d721fa629f80394a3effe505ecb0c0ba3aaf8934b2f76b8054dff9cd92a2","textHash":"f65d34a7fc42cbeaf2f7e50dea62871d37ac714d7c1ef5db4acf8dfdd5254da3","observedAt":"2026-11-19T09:48:10.304Z"}},{"actor":{"name":"Yang, Xinyu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yang, Xinyu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24877","locator":"head/meta[name=author|citation_author]"},"assertionId":"ca0261966964b278f3f5218a7ce1f8dc99b5fdde2f2a7d0b9df91fef3f5947b5:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T09:48:10.304Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ca0261966964b278f3f5218a7ce1f8dc99b5fdde2f2a7d0b9df91fef3f5947b5","sourceUrl":"https://arxiv.org/abs/2611.24877","contentSha256":"56d4d721fa629f80394a3effe505ecb0c0ba3aaf8934b2f76b8054dff9cd92a2","textHash":"f65d34a7fc42cbeaf2f7e50dea62871d37ac714d7c1ef5db4acf8dfdd5254da3","observedAt":"2026-11-19T09:48:10.304Z"}}],"participationCoverage":{"total":2,"returned":2,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.24877","snapshotId":"ca0261966964b278f3f5218a7ce1f8dc99b5fdde2f2a7d0b9df91fef3f5947b5","contentSha256":"56d4d721fa629f80394a3effe505ecb0c0ba3aaf8934b2f76b8054dff9cd92a2","textHash":"f65d34a7fc42cbeaf2f7e50dea62871d37ac714d7c1ef5db4acf8dfdd5254da3","observedAt":"2026-11-19T09:48:10.304Z","basis":"publisher-page-metadata","sourceLinks":42,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/d6aab00c76f20543b21c6c2c2e4b191ee8913936045c076b78025e9daff32e5a/source"}}}],"audits":[{"publicationId":"8f16308d-bce7-447f-8a94-a9c91b554b6f","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading correctly attributes the study to Zhiyu Liu and Xinyu Yang and keeps the results and conclusion as the authors' reported findings.","excerpt":"Authors:Zhiyu Liu, Xinyu Yang"},{"dimension":"summary","status":"uncertain","explanation":"The reading is mostly faithful, but it gives the 11% discovery rate without the abstract's comparators: random pairs 6%, a regression scan 28%, and 33% vs 30% after a training-data check. Without them, readers cannot judge the result; the smallest fix is to add the baselines. It drops 'post hoc' from the small-steps finding and the word 'faithfully' from the authors' conclusion. It also places 'in this test' inside the authors' conclusion, which the abstract does not do. Central terms such as surrogate model, black-box and 'randomised later' are not explained for non-specialists.","excerpt":"11% held in patients randomised later (random pairs 6%; a regression scan's shorter list of its strongest candidates, 28%); after a check on the training data, 33% against 30%"},{"dimension":"date","status":"pass","explanation":"The originalPublished date of 2026-11-18 matches the v1 submission date on arXiv.","excerpt":"[Submitted on 18 Nov 2026]"},{"dimension":"support","status":"uncertain","explanation":"The source-stated claims match the abstract. The claim 'Only 11% of 601 ... held' adds evaluative framing and omits the 6% random baseline, which the LLM exceeded, so 'Only' is not supported as stated. The interpretation claim about scope is reasonably conditional, but its excerpt is the authors' positive conclusion, which does not itself support the limitation. Dropping 'Only' or adding the comparators would repair the claim.","excerpt":"Of 601 unexpected relationships it read out of the clean models, 11% held in patients randomised later (random pairs 6%"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper, which is the most relevant way to access the source.","excerpt":null}],"createdAt":"2026-11-19T09:17:42.604Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}