# arXiv cs.AI: Researchers introduce Synthetic Hospital, a synthetic benchmark for clinical AI [Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-researchers-introduce-synthetic-hospital-a-d284d3ac) The authors introduce Synthetic Hospital, an open, fully synthetic benchmark of longitudinal EHRs, designed to test whether AI models can track patient information across encounters. They argue that real EHR data is difficult to share openly and lacks verifiable ground truth: records show what clinicians documented, not necessarily everything that happened. Synthetic Hospital contains 1,268 patients and 5,602 encounters. Its diagnoses, findings and temporal relations are grounded in standard ontologies, with provenance linking them to public medical-education material. A simulated hospital record system provides access through interfaces that mirror real EHR infrastructure. In a blinded review, physicians identified the benchmark’s records as synthetic or real at near-chance rates. Yet the authors report that even the best model misses roughly half of clinically relevant findings when summarizing a chart. That model matched the mean of seven physicians on a matched subset when reconstructing longitudinal problem lists, but scored below the best physician. The authors present these results as evidence that the benchmark is a difficult test of clinical AI performance. The abstract does not establish how performance on this synthetic benchmark relates to performance in real clinical workflows. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Sep 24, 2026 (2026-09-24). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-14T05:05:39.816Z. First observed: 2026-09-25T07:40:35.728Z. Retrieved: 2026-10-30T21:31:38.023Z. - [Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmark](https://arxiv.org/abs/2609.30027) ## Limits The abstract reports benchmark results but does not establish how performance on synthetic records predicts performance in real clinical workflows. The abstract does not describe the benchmark's coverage of patient populations or clinical settings; its generalizability beyond the reported evaluation is therefore unclear. ## Open question How well does performance on Synthetic Hospital predict model performance on real patient records and in clinical workflows? ## Source-stated The authors introduce Synthetic Hospital as an open, fully synthetic benchmark for longitudinal EHRs, intended to address barriers to sharing real records and establishing verifiable ground truth. > We introduce Synthetic Hospital, an open, fully synthetic, fact-grounded longitudinal EHR benchmark that resolves the open sharing and verifiable ground truth barriers. - [Supporting source](https://arxiv.org/abs/2609.30027) ## Source-stated The benchmark contains 1,268 longitudinal patients and 5,602 encounters, grounded in ontologies and linked to public medical-education material. > it comprises 1,268 longitudinal patients and 5,602 encounters, where every diagnosis, finding, and temporal relation is grounded in standard ontologies - [Supporting source](https://arxiv.org/abs/2609.30027) ## Source-stated Physicians distinguished the benchmark's records from real patient charts at near-chance rates in a blinded review. > In a blinded review, physicians distinguished its records from real patient charts at near-chance rates (53%). - [Supporting source](https://arxiv.org/abs/2609.30027) ## Source-stated The best model matched the mean of seven physicians on a matched subset for reconstructing a longitudinal problem list, but fell below the best physician. > the best model reconstructs a patient's longitudinal problem list with a severity-weighted F1 of 0.73, level with the mean of seven physicians on a matched subset but well below the best of them (0.89) - [Supporting source](https://arxiv.org/abs/2609.30027) ## Source-stated The authors report that models miss roughly half of clinically relevant findings when summarizing charts. > misses roughly half of clinically relevant findings when summarizing a chart - [Supporting source](https://arxiv.org/abs/2609.30027) ## Source and contributor context Park, Christine, Chen, Valerie, Dettmers, Tim · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 07cc7b09-ec20-45e0-8288-c732ce478df9. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-14T05:05:39.816Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 0a2a28ce773e4370d3e29c27f3f6e47974c28d830f5a0d91556b0861a7d549dd. Source text hash: e351d1201e997647059a5b3e42ef66ec9a415484c9ae395e094c3d9ee25da52e. ## Revision and source-check history ### Version 1 — 2026-11-14T05:05:39.816Z The authors introduce Synthetic Hospital, an open, fully synthetic benchmark of longitudinal EHRs, designed to test whether AI models can track patient information across encounters. They argue that real EHR data is difficult to share openly and lacks verifiable ground truth: records show what clinicians documented, not necessarily everything that happened. Synthetic Hospital contains 1,268 patients and 5,602 encounters. Its diagnoses, findings and temporal relations are grounded in standard ontologies, with provenance linking them to public medical-education material. A simulated hospital record system provides access through interfaces that mirror real EHR infrastructure. In a blinded review, physicians identified the benchmark’s records as synthetic or real at near-chance rates. Yet the authors report that even the best model misses roughly half of clinically relevant findings when summarizing a chart. That model matched the mean of seven physicians on a matched subset when reconstructing longitudinal problem lists, but scored below the best physician. The authors present these results as evidence that the benchmark is a difficult test of clinical AI performance. The abstract does not establish how performance on this synthetic benchmark relates to performance in real clinical workflows. ## Original source arXiv cs.AI · Article Original source date: Sep 24, 2026 (2026-09-24). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-14T05:05:39.816Z. First observed: 2026-09-25T07:40:35.728Z. Retrieved: 2026-10-30T21:31:38.023Z. - [Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmark](https://arxiv.org/abs/2609.30027) ## Limits The abstract reports benchmark results but does not establish how performance on synthetic records predicts performance in real clinical workflows. The abstract does not describe the benchmark's coverage of patient populations or clinical settings; its generalizability beyond the reported evaluation is therefore unclear. ## Open question How well does performance on Synthetic Hospital predict model performance on real patient records and in clinical workflows? ## Source-stated The authors introduce Synthetic Hospital as an open, fully synthetic benchmark for longitudinal EHRs, intended to address barriers to sharing real records and establishing verifiable ground truth. > We introduce Synthetic Hospital, an open, fully synthetic, fact-grounded longitudinal EHR benchmark that resolves the open sharing and verifiable ground truth barriers. - [Supporting source](https://arxiv.org/abs/2609.30027) ## Source-stated The benchmark contains 1,268 longitudinal patients and 5,602 encounters, grounded in ontologies and linked to public medical-education material. > it comprises 1,268 longitudinal patients and 5,602 encounters, where every diagnosis, finding, and temporal relation is grounded in standard ontologies - [Supporting source](https://arxiv.org/abs/2609.30027) ## Source-stated Physicians distinguished the benchmark's records from real patient charts at near-chance rates in a blinded review. > In a blinded review, physicians distinguished its records from real patient charts at near-chance rates (53%). - [Supporting source](https://arxiv.org/abs/2609.30027) ## Source-stated The best model matched the mean of seven physicians on a matched subset for reconstructing a longitudinal problem list, but fell below the best physician. > the best model reconstructs a patient's longitudinal problem list with a severity-weighted F1 of 0.73, level with the mean of seven physicians on a matched subset but well below the best of them (0.89) - [Supporting source](https://arxiv.org/abs/2609.30027) ## Source-stated The authors report that models miss roughly half of clinically relevant findings when summarizing charts. > misses roughly half of clinically relevant findings when summarizing a chart - [Supporting source](https://arxiv.org/abs/2609.30027) ## Source and contributor context Park, Christine, Chen, Valerie, Dettmers, Tim · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 07cc7b09-ec20-45e0-8288-c732ce478df9. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-14T05:05:39.816Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 0a2a28ce773e4370d3e29c27f3f6e47974c28d830f5a0d91556b0861a7d549dd. Source text hash: e351d1201e997647059a5b3e42ef66ec9a415484c9ae395e094c3d9ee25da52e. ### uncertain — 2026-11-14T05:05:39.816Z — anthropic/claude-opus-5.5 Publication: 07cc7b09-ec20-45e0-8288-c732ce478df9 attribution — pass: The draft attributes the benchmark, its design rationale and its results to the paper's authors (Park, Chen, Dettmers) on arXiv cs.AI. The 'difficult test' conclusion is correctly framed as the authors' own presentation, not as an established fact. > Authors:Christine Park, Valerie Chen, Tim Dettmers summary — uncertain: Two gaps affect a lay reader. First, 'EHR' is never expanded to 'electronic health record', though it is the central term; add the expansion or a tooltip. Second, the draft omits the authors' framing of why this matters: frontier models are rarely used clinically because realistic benchmarks are scarce. One clause can restore it. It also softens 'well below the best' to 'below' and drops 'none approaches ceiling' and the 0.73 vs 0.89 gap. Smallest repair: restore 'well below' and note the result covers 10 models. > Frontier language models are rarely used in clinical workflows because the realistic, longitudinal benchmarks needed to develop them are scarce. date — pass: originalPublished 2026-09-24 matches the v1 submission date, not the later v2 revision (27 Oct) or the feed date (29 Oct). > \[Submitted on 24 Sep 2026 (v1), last revised 27 Oct 2026 (this version, v2)\] support — pass: Counts, grounding, blinded review and model-versus-physician comparisons match the abstract. Claim 5 says 'models miss roughly half', but the abstract attributes this to the best model. This is a minor broadening; the summary body states it correctly. The limitations and the question are appropriately conditional and do not overstate real-world reliability. > misses roughly half of clinically relevant findings when summarizing a chart media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/d284d3acaf2c34b26f87f9842fabe357ef2a7f8e936e7e8fda9460c002d4900a/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-researchers-introduce-synthetic-hospital-a-d284d3ac.json)