{"id":"c777657e62a0012ef756da6d7d54226f4ec8fddde3c53fbb09ed15cceae09d6a","title":"MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine","argument":"The authors of MedJev report evaluating an open System One model that runs locally on a laptop and answers structured questions in a single forward pass, without generating text. Across ten medical benchmarks, they report competitive results on medical knowledge and diagnosis tasks, but a more mixed picture for patient-level prediction: MedJev exceeded XGBoost on In Hospital Mortality AUROC, while underperforming supervised models on ED Triage Admission. The authors also report substantial miscalibration on In Hospital Mortality, meaning its confidence estimates did not align well with observed outcomes. Their lightweight recalibration method uses labeled examples from a local patient population to reduce that mismatch. The results make MedJev a research baseline for examining whether locally run models can support medical prediction at low computational cost; the abstract does not establish clinical effectiveness. How well does the reported performance and recalibration hold up across hospitals and in clinical use?","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the arXiv abstract, not the full paper, so it does not provide details of benchmark methods or results beyond those stated there. The abstract reports benchmark evaluations, not clinical effectiveness in use. The abstract does not establish how performance or recalibration generalizes across hospitals.","speaker":{"name":"Lee, Simon A., Sha, Jiahang, Jeong, Moonseong, Pan, Yunbei, Chiang, Jeffrey N.","roleAtSource":"Article author"},"source":{"title":"MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.36006","publishedAt":"2026-10-26","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors describe System One decision models as answering structured questions in a single forward pass without generating text, and say MedJev runs locally on a laptop.","excerpt":"answer structured questions in a single forward pass without generating text","url":"https://arxiv.org/abs/2610.36006","relationship":"Source-stated"},{"text":"The authors evaluated MedJev across ten medical benchmarks spanning medical knowledge, diagnosis, and patient-level prediction.","excerpt":"across ten medical benchmarks spanning medical knowledge, diagnosis, and patient level prediction","url":"https://arxiv.org/abs/2610.36006","relationship":"Source-stated"},{"text":"The authors report 72.5 percent accuracy on MedQA, 75.4 percent on USMLE, and 80.1 percent on NEJM.","excerpt":"accuracies of 72.5 percent on MedQA, 75.4 percent on USMLE, and 80.1 percent on NEJM","url":"https://arxiv.org/abs/2610.36006","relationship":"Source-stated"},{"text":"For In Hospital Mortality, the authors report an AUROC of 0.926 for MedJev, compared with 0.902 for XGBoost.","excerpt":"AUROC of 0.926 for In Hospital Mortality, exceeding XGBoost (0.902)","url":"https://arxiv.org/abs/2610.36006","relationship":"Source-stated"},{"text":"For ED Triage Admission, the authors report that MedJev underperformed supervised models, with results of 0.745 versus 0.809.","excerpt":"underperforms supervised models on operational outcomes such as ED Triage Admission (0.745 vs. 0.809)","url":"https://arxiv.org/abs/2610.36006","relationship":"Source-stated"},{"text":"The abstract reports benchmark results and a recalibration method; it does not report evidence of clinical effectiveness in use.","excerpt":"We evaluate MedJev, an open System One model that runs locally on a laptop","url":"https://arxiv.org/abs/2610.36006","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36006","checkedAt":"2026-11-10T21:04:11.348Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well does the reported performance and recalibration hold up across hospitals and in clinical use?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine","summary":"The authors of MedJev report evaluating an open System One model that runs locally on a laptop and answers structured questions in a single forward pass, without generating text. Across ten medical benchmarks, they report competitive results on medical knowledge and diagnosis tasks, but a more mixed picture for patient-level prediction: MedJev exceeded XGBoost on In Hospital Mortality AUROC, while underperforming supervised models on ED Triage Admission. The authors also report substantial miscalibration on In Hospital Mortality, meaning its confidence estimates did not align well with observed outcomes. Their lightweight recalibration method uses labeled examples from a local patient population to reduce that mismatch. The results make MedJev a research baseline for examining whether locally run models can support medical prediction at low computational cost; the abstract does not establish clinical effectiveness. How well does the reported performance and recalibration hold up across hospitals and in clinical use?","question":"How well does the reported performance and recalibration hold up across hospitals and in clinical use?","claims":[{"text":"The authors describe System One decision models as answering structured questions in a single forward pass without generating text, and say MedJev runs locally on a laptop.","excerpt":"answer structured questions in a single forward pass without generating text","support":"source-stated"},{"text":"The authors evaluated MedJev across ten medical benchmarks spanning medical knowledge, diagnosis, and patient-level prediction.","excerpt":"across ten medical benchmarks spanning medical knowledge, diagnosis, and patient level prediction","support":"source-stated"},{"text":"The authors report 72.5 percent accuracy on MedQA, 75.4 percent on USMLE, and 80.1 percent on NEJM.","excerpt":"accuracies of 72.5 percent on MedQA, 75.4 percent on USMLE, and 80.1 percent on NEJM","support":"source-stated"},{"text":"For In Hospital Mortality, the authors report an AUROC of 0.926 for MedJev, compared with 0.902 for XGBoost.","excerpt":"AUROC of 0.926 for In Hospital Mortality, exceeding XGBoost (0.902)","support":"source-stated"},{"text":"For ED Triage Admission, the authors report that MedJev underperformed supervised models, with results of 0.745 versus 0.809.","excerpt":"underperforms supervised models on operational outcomes such as ED Triage Admission (0.745 vs. 0.809)","support":"source-stated"},{"text":"The abstract reports benchmark results and a recalibration method; it does not report evidence of clinical effectiveness in use.","excerpt":"We evaluate MedJev, an open System One model that runs locally on a laptop","support":"interpretation"}],"limitations":["The supplied evidence is the arXiv abstract, not the full paper, so it does not provide details of benchmark methods or results beyond those stated there.","The abstract reports benchmark evaluations, not clinical effectiveness in use.","The abstract does not establish how performance or recalibration generalizes across hospitals."],"mediaId":"1b80831f56690d43700929f0f8c74f32451621a2a2576c98c6302e39e4704ed1","correctionReason":null,"id":"c777657e62a0012ef756da6d7d54226f4ec8fddde3c53fbb09ed15cceae09d6a","publicationId":"71180004-0eab-4ac7-8be8-ee8a90ab2c7b","version":1,"sourceUrl":"https://arxiv.org/abs/2610.36006","sourceTitle":"MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-26","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-28T07:28:45.934Z","createdAt":"2026-11-10T20:33:01.515Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"1b80831f56690d43700929f0f8c74f32451621a2a2576c98c6302e39e4704ed1","kind":"article","url":"https://arxiv.org/abs/2610.36006","sourceUrl":"https://arxiv.org/abs/2610.36006","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-30T21:03:43.022Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36006","checkedAt":"2026-11-10T21:04:11.348Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"1b80831f56690d43700929f0f8c74f32451621a2a2576c98c6302e39e4704ed1","confidence":0.96,"provider":"typesafe","model":"jev-1.13.0","inputHash":"9c3407e6efcabc5b8890308774369d73381e1c6d2e17545818b67ae5262c12da","decidedAt":"2026-10-30T21:03:45.813Z"},"provenance":{"textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observationId":"612094e9188f4cc561909c0299cc786d30a30baa7fa05eee5fb4cd6ee3d8d81c","retrievedAt":"2026-10-30T21:03:42.742Z","kind":"publisher-page","preparer":{"model":"server-excerpt-repair/stream-excerpt-repair-v1 (original draft: gpt-6-luna)","runId":"fa8da14d-48e9-4430-8389-ad21556a6534"},"reviewer":{"model":"gpt-6-luna","runId":"9480b1bb-79ac-4ddb-8367-2e4c10b55d85"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T20:33:01.515Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Lee, Simon A.","url":null,"kind":"unknown"},{"name":"Sha, Jiahang","url":null,"kind":"unknown"},{"name":"Jeong, Moonseong","url":null,"kind":"unknown"},{"name":"Pan, Yunbei","url":null,"kind":"unknown"},{"name":"Chiang, Jeffrey N.","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Lee, Simon A.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Lee, Simon A.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36006","locator":"head/meta[name=author|citation_author]"},"assertionId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T21:04:11.348Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e","sourceUrl":"https://arxiv.org/abs/2610.36006","contentSha256":"0e0079b3ef98fa0bcbb6c5fa687a0bbc4d241e3634b324b32fb38a969357269c","textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observedAt":"2026-11-10T21:04:11.348Z"}},{"actor":{"name":"Sha, Jiahang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sha, Jiahang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36006","locator":"head/meta[name=author|citation_author]"},"assertionId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T21:04:11.348Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e","sourceUrl":"https://arxiv.org/abs/2610.36006","contentSha256":"0e0079b3ef98fa0bcbb6c5fa687a0bbc4d241e3634b324b32fb38a969357269c","textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observedAt":"2026-11-10T21:04:11.348Z"}},{"actor":{"name":"Jeong, Moonseong","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jeong, Moonseong","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36006","locator":"head/meta[name=author|citation_author]"},"assertionId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T21:04:11.348Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e","sourceUrl":"https://arxiv.org/abs/2610.36006","contentSha256":"0e0079b3ef98fa0bcbb6c5fa687a0bbc4d241e3634b324b32fb38a969357269c","textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observedAt":"2026-11-10T21:04:11.348Z"}},{"actor":{"name":"Pan, Yunbei","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Pan, Yunbei","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36006","locator":"head/meta[name=author|citation_author]"},"assertionId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T21:04:11.348Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e","sourceUrl":"https://arxiv.org/abs/2610.36006","contentSha256":"0e0079b3ef98fa0bcbb6c5fa687a0bbc4d241e3634b324b32fb38a969357269c","textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observedAt":"2026-11-10T21:04:11.348Z"}},{"actor":{"name":"Chiang, Jeffrey N.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chiang, Jeffrey N.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36006","locator":"head/meta[name=author|citation_author]"},"assertionId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T21:04:11.348Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e","sourceUrl":"https://arxiv.org/abs/2610.36006","contentSha256":"0e0079b3ef98fa0bcbb6c5fa687a0bbc4d241e3634b324b32fb38a969357269c","textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observedAt":"2026-11-10T21:04:11.348Z"}}],"participationCoverage":{"total":5,"returned":5,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.36006","snapshotId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e","contentSha256":"0e0079b3ef98fa0bcbb6c5fa687a0bbc4d241e3634b324b32fb38a969357269c","textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observedAt":"2026-11-10T21:04:11.348Z","basis":"publisher-page-metadata","sourceLinks":45,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/c777657e62a0012ef756da6d7d54226f4ec8fddde3c53fbb09ed15cceae09d6a/source"}},"canonicalUrl":"https://escapemeridian.com/readings/medjev-evaluating-zero-shot-system-one-decision-making-in-c777657e","formats":{"html":"/readings/medjev-evaluating-zero-shot-system-one-decision-making-in-c777657e","markdown":"/readings/medjev-evaluating-zero-shot-system-one-decision-making-in-c777657e.md","json":"/readings/medjev-evaluating-zero-shot-system-one-decision-making-in-c777657e.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-zero-shot","name":"Zero-shot","explanation":"Doing a task without worked examples (zero-shot) or with only a few examples given in the prompt (few-shot).","referenceUrl":"https://developers.google.com/machine-learning/glossary#zero-shot-prompting"},{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"c777657e62a0012ef756da6d7d54226f4ec8fddde3c53fbb09ed15cceae09d6a","revision":"71180004-0eab-4ac7-8be8-ee8a90ab2c7b","revisionKind":"publication-id","title":"MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine","url":"https://escapemeridian.com/readings/medjev-evaluating-zero-shot-system-one-decision-making-in-c777657e"},"source":{"type":"source","id":"https://arxiv.org/abs/2610.36006","revision":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","revisionKind":"sha256-source-text","title":"MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine","url":"https://arxiv.org/abs/2610.36006"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"c777657e62a0012ef756da6d7d54226f4ec8fddde3c53fbb09ed15cceae09d6a","title":"MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine","argument":"The authors of MedJev report evaluating an open System One model that runs locally on a laptop and answers structured questions in a single forward pass, without generating text. Across ten medical benchmarks, they report competitive results on medical knowledge and diagnosis tasks, but a more mixed picture for patient-level prediction: MedJev exceeded XGBoost on In Hospital Mortality AUROC, while underperforming supervised models on ED Triage Admission. The authors also report substantial miscalibration on In Hospital Mortality, meaning its confidence estimates did not align well with observed outcomes. Their lightweight recalibration method uses labeled examples from a local patient population to reduce that mismatch. The results make MedJev a research baseline for examining whether locally run models can support medical prediction at low computational cost; the abstract does not establish clinical effectiveness. How well does the reported performance and recalibration hold up across hospitals and in clinical use?","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the arXiv abstract, not the full paper, so it does not provide details of benchmark methods or results beyond those stated there. The abstract reports benchmark evaluations, not clinical effectiveness in use. The abstract does not establish how performance or recalibration generalizes across hospitals.","speaker":{"name":"Lee, Simon A., Sha, Jiahang, Jeong, Moonseong, Pan, Yunbei, Chiang, Jeffrey N.","roleAtSource":"Article author"},"source":{"title":"MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.36006","publishedAt":"2026-10-26","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors describe System One decision models as answering structured questions in a single forward pass without generating text, and say MedJev runs locally on a laptop.","excerpt":"answer structured questions in a single forward pass without generating text","url":"https://arxiv.org/abs/2610.36006","relationship":"Source-stated"},{"text":"The authors evaluated MedJev across ten medical benchmarks spanning medical knowledge, diagnosis, and patient-level prediction.","excerpt":"across ten medical benchmarks spanning medical knowledge, diagnosis, and patient level prediction","url":"https://arxiv.org/abs/2610.36006","relationship":"Source-stated"},{"text":"The authors report 72.5 percent accuracy on MedQA, 75.4 percent on USMLE, and 80.1 percent on NEJM.","excerpt":"accuracies of 72.5 percent on MedQA, 75.4 percent on USMLE, and 80.1 percent on NEJM","url":"https://arxiv.org/abs/2610.36006","relationship":"Source-stated"},{"text":"For In Hospital Mortality, the authors report an AUROC of 0.926 for MedJev, compared with 0.902 for XGBoost.","excerpt":"AUROC of 0.926 for In Hospital Mortality, exceeding XGBoost (0.902)","url":"https://arxiv.org/abs/2610.36006","relationship":"Source-stated"},{"text":"For ED Triage Admission, the authors report that MedJev underperformed supervised models, with results of 0.745 versus 0.809.","excerpt":"underperforms supervised models on operational outcomes such as ED Triage Admission (0.745 vs. 0.809)","url":"https://arxiv.org/abs/2610.36006","relationship":"Source-stated"},{"text":"The abstract reports benchmark results and a recalibration method; it does not report evidence of clinical effectiveness in use.","excerpt":"We evaluate MedJev, an open System One model that runs locally on a laptop","url":"https://arxiv.org/abs/2610.36006","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36006","checkedAt":"2026-11-10T21:04:11.348Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well does the reported performance and recalibration hold up across hospitals and in clinical use?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine","summary":"The authors of MedJev report evaluating an open System One model that runs locally on a laptop and answers structured questions in a single forward pass, without generating text. Across ten medical benchmarks, they report competitive results on medical knowledge and diagnosis tasks, but a more mixed picture for patient-level prediction: MedJev exceeded XGBoost on In Hospital Mortality AUROC, while underperforming supervised models on ED Triage Admission. The authors also report substantial miscalibration on In Hospital Mortality, meaning its confidence estimates did not align well with observed outcomes. Their lightweight recalibration method uses labeled examples from a local patient population to reduce that mismatch. The results make MedJev a research baseline for examining whether locally run models can support medical prediction at low computational cost; the abstract does not establish clinical effectiveness. How well does the reported performance and recalibration hold up across hospitals and in clinical use?","question":"How well does the reported performance and recalibration hold up across hospitals and in clinical use?","claims":[{"text":"The authors describe System One decision models as answering structured questions in a single forward pass without generating text, and say MedJev runs locally on a laptop.","excerpt":"answer structured questions in a single forward pass without generating text","support":"source-stated"},{"text":"The authors evaluated MedJev across ten medical benchmarks spanning medical knowledge, diagnosis, and patient-level prediction.","excerpt":"across ten medical benchmarks spanning medical knowledge, diagnosis, and patient level prediction","support":"source-stated"},{"text":"The authors report 72.5 percent accuracy on MedQA, 75.4 percent on USMLE, and 80.1 percent on NEJM.","excerpt":"accuracies of 72.5 percent on MedQA, 75.4 percent on USMLE, and 80.1 percent on NEJM","support":"source-stated"},{"text":"For In Hospital Mortality, the authors report an AUROC of 0.926 for MedJev, compared with 0.902 for XGBoost.","excerpt":"AUROC of 0.926 for In Hospital Mortality, exceeding XGBoost (0.902)","support":"source-stated"},{"text":"For ED Triage Admission, the authors report that MedJev underperformed supervised models, with results of 0.745 versus 0.809.","excerpt":"underperforms supervised models on operational outcomes such as ED Triage Admission (0.745 vs. 0.809)","support":"source-stated"},{"text":"The abstract reports benchmark results and a recalibration method; it does not report evidence of clinical effectiveness in use.","excerpt":"We evaluate MedJev, an open System One model that runs locally on a laptop","support":"interpretation"}],"limitations":["The supplied evidence is the arXiv abstract, not the full paper, so it does not provide details of benchmark methods or results beyond those stated there.","The abstract reports benchmark evaluations, not clinical effectiveness in use.","The abstract does not establish how performance or recalibration generalizes across hospitals."],"mediaId":"1b80831f56690d43700929f0f8c74f32451621a2a2576c98c6302e39e4704ed1","correctionReason":null,"id":"c777657e62a0012ef756da6d7d54226f4ec8fddde3c53fbb09ed15cceae09d6a","publicationId":"71180004-0eab-4ac7-8be8-ee8a90ab2c7b","version":1,"sourceUrl":"https://arxiv.org/abs/2610.36006","sourceTitle":"MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-26","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-28T07:28:45.934Z","createdAt":"2026-11-10T20:33:01.515Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"1b80831f56690d43700929f0f8c74f32451621a2a2576c98c6302e39e4704ed1","kind":"article","url":"https://arxiv.org/abs/2610.36006","sourceUrl":"https://arxiv.org/abs/2610.36006","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-30T21:03:43.022Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36006","checkedAt":"2026-11-10T21:04:11.348Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"1b80831f56690d43700929f0f8c74f32451621a2a2576c98c6302e39e4704ed1","confidence":0.96,"provider":"typesafe","model":"jev-1.13.0","inputHash":"9c3407e6efcabc5b8890308774369d73381e1c6d2e17545818b67ae5262c12da","decidedAt":"2026-10-30T21:03:45.813Z"},"provenance":{"textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observationId":"612094e9188f4cc561909c0299cc786d30a30baa7fa05eee5fb4cd6ee3d8d81c","retrievedAt":"2026-10-30T21:03:42.742Z","kind":"publisher-page","preparer":{"model":"server-excerpt-repair/stream-excerpt-repair-v1 (original draft: gpt-6-luna)","runId":"fa8da14d-48e9-4430-8389-ad21556a6534"},"reviewer":{"model":"gpt-6-luna","runId":"9480b1bb-79ac-4ddb-8367-2e4c10b55d85"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T20:33:01.515Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Lee, Simon A.","url":null,"kind":"unknown"},{"name":"Sha, Jiahang","url":null,"kind":"unknown"},{"name":"Jeong, Moonseong","url":null,"kind":"unknown"},{"name":"Pan, Yunbei","url":null,"kind":"unknown"},{"name":"Chiang, Jeffrey N.","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Lee, Simon A.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Lee, Simon A.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36006","locator":"head/meta[name=author|citation_author]"},"assertionId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T21:04:11.348Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e","sourceUrl":"https://arxiv.org/abs/2610.36006","contentSha256":"0e0079b3ef98fa0bcbb6c5fa687a0bbc4d241e3634b324b32fb38a969357269c","textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observedAt":"2026-11-10T21:04:11.348Z"}},{"actor":{"name":"Sha, Jiahang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sha, Jiahang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36006","locator":"head/meta[name=author|citation_author]"},"assertionId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T21:04:11.348Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e","sourceUrl":"https://arxiv.org/abs/2610.36006","contentSha256":"0e0079b3ef98fa0bcbb6c5fa687a0bbc4d241e3634b324b32fb38a969357269c","textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observedAt":"2026-11-10T21:04:11.348Z"}},{"actor":{"name":"Jeong, Moonseong","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jeong, Moonseong","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36006","locator":"head/meta[name=author|citation_author]"},"assertionId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T21:04:11.348Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e","sourceUrl":"https://arxiv.org/abs/2610.36006","contentSha256":"0e0079b3ef98fa0bcbb6c5fa687a0bbc4d241e3634b324b32fb38a969357269c","textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observedAt":"2026-11-10T21:04:11.348Z"}},{"actor":{"name":"Pan, Yunbei","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Pan, Yunbei","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36006","locator":"head/meta[name=author|citation_author]"},"assertionId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T21:04:11.348Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e","sourceUrl":"https://arxiv.org/abs/2610.36006","contentSha256":"0e0079b3ef98fa0bcbb6c5fa687a0bbc4d241e3634b324b32fb38a969357269c","textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observedAt":"2026-11-10T21:04:11.348Z"}},{"actor":{"name":"Chiang, Jeffrey N.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chiang, Jeffrey N.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36006","locator":"head/meta[name=author|citation_author]"},"assertionId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T21:04:11.348Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e","sourceUrl":"https://arxiv.org/abs/2610.36006","contentSha256":"0e0079b3ef98fa0bcbb6c5fa687a0bbc4d241e3634b324b32fb38a969357269c","textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observedAt":"2026-11-10T21:04:11.348Z"}}],"participationCoverage":{"total":5,"returned":5,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.36006","snapshotId":"86e1ad794629539198ece22c826c3c2e653a0462ae3189f98e032278afac7d4e","contentSha256":"0e0079b3ef98fa0bcbb6c5fa687a0bbc4d241e3634b324b32fb38a969357269c","textHash":"e591f52ef3e0deb9a46ab0c360d544bc65d7f0ae6879b996b238276cf65aaced","observedAt":"2026-11-10T21:04:11.348Z","basis":"publisher-page-metadata","sourceLinks":45,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/c777657e62a0012ef756da6d7d54226f4ec8fddde3c53fbb09ed15cceae09d6a/source"}}}],"audits":[{"publicationId":"71180004-0eab-4ac7-8be8-ee8a90ab2c7b","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading attributes the evaluation, results and recalibration method to the MedJev paper's authors. The authors are listed on the arXiv page, and the reading does not present their claims as established fact.","excerpt":"Authors:Simon A. Lee, Jiahang Sha, Moonseong Jeong, Yunbei Pan, Jeffrey N. Chiang"},{"dimension":"summary","status":"uncertain","explanation":"The reading is broadly faithful but has three fidelity gaps. First, it drops the authors' central benchmark claim that MedJev approaches published GPT-4 performance with well-calibrated knowledge-task confidence. Second, it softens their conclusion that open System One models are a promising zero-shot baseline into an unattributed EM framing. Third, AUROC, XGBoost and zero-shot are central but unexplained for lay readers. Smallest repair: restore the attributed GPT-4 comparison and the authors' promising-baseline conclusion, and add term tooltips.","excerpt":"approaching published GPT 4 performance while maintaining well calibrated confidence estimates (ECE 0.028 to 0.063)"},{"dimension":"date","status":"pass","explanation":"The originalPublished date, 2026-10-26, matches the v1 submission date on arXiv. The feed date is later and is correctly not used.","excerpt":"[Submitted on 26 Oct 2026]"},{"dimension":"support","status":"pass","explanation":"The figures, comparisons, miscalibration finding and recalibration description all match the abstract. The labeled interpretation, that the abstract shows benchmark results rather than clinical effectiveness, is a fair limitation. The question about generalization across hospitals follows from the local-recalibration design. The word competitive for diagnosis tasks is loosely grounded but not contradicted.","excerpt":"reducing ECE to 0.007"},{"dimension":"media","status":"pass","explanation":"The selected media item is the verified direct arXiv abstract page for this paper, which is the most relevant way to access the source.","excerpt":null}],"createdAt":"2026-11-10T20:33:01.515Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}