# arXiv cs.AI: Nusrat and Thind introduce OpenGray, a radiotherapy planning benchmark [Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-nusrat-and-thind-introduce-opengray-a-radiotherapy-a1e392ff) Humza Nusrat and Kundan Thind present OpenGray, a benchmark designed to test whether radiotherapy planning agents make appropriate decisions, not just whether their plans achieve favorable dose results. It adds tasks with known correct decisions to the OpenKBP-Opt head-and-neck dataset, including cases where dose goals cannot all be met and a feasible control with a stored plan meeting every hard goal. Across 30 patients, systems using Terra had high decision accuracy overall, but escalated only some contradictions that required arithmetic on the anatomy. All four systems submitted the feasible control, yet 24% of those plans missed a hard goal met by the stored plan. In separate stereotactic radiosurgery runs, SAGE identified a goal that failed at every attempt, but had no action for stopping. The authors argue that aggregate scores can conceal important decision failures. Their findings suggest that evaluating dose alone may miss whether an agent recognizes an infeasible task, requests review, or stops. The reported results come from the benchmark and runs described in the paper; they do not establish how these agents would perform in broader clinical use. How well would decision-focused evaluation predict safe behavior in clinical settings? Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Oct 27, 2026 (2026-10-27). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-14T03:01:55.209Z. First observed: 2026-10-28T05:52:45.035Z. Retrieved: 2026-10-30T06:16:22.820Z. - [OpenGray: a benchmark for evaluating the decisions of radiotherapy planning agents](https://arxiv.org/abs/2610.37081) ## Limits The evaluation covered 30 patients in the benchmark and logs from 39 stereotactic radiosurgery runs; the evidence does not establish performance in broader clinical use. The abstract reports selected results but does not provide enough detail to assess how the findings generalize beyond the evaluated tasks and systems. ## Open question How well would decision-focused evaluation predict safe behavior in clinical settings? ## Source-stated Nusrat and Thind developed OpenGray to evaluate radiotherapy planning agents on tasks with known correct decisions, in addition to dose. > This study developed OpenGray, a benchmark of planning tasks with known correct decisions. - [Supporting source](https://arxiv.org/abs/2610.37081) ## Source-stated In the study's 30-patient evaluation, systems using Terra achieved high overall decision accuracy but escalated only a portion of overlap contradictions. > decision accuracies of 95.4% and 92.1%, respectively, yet escalated only 50% (95% confidence interval, 25-75%) and 25% (10-40%) of overlap contradictions - [Supporting source](https://arxiv.org/abs/2610.37081) ## Source-stated All four evaluated systems submitted the feasible control, but 24% of those plans missed a hard goal met by the stored plan. > All four systems submitted every feasible control, but 24% of these plans missed a hard goal that the stored plan had met. - [Supporting source](https://arxiv.org/abs/2610.37081) ## Source-stated In four SAGE runs, the agent identified a repeatedly failing goal but had no action for stopping. > In four SAGE runs, one goal failed at every attempt; the agent identified it in every reply but had no action for stopping. - [Supporting source](https://arxiv.org/abs/2610.37081) ## Source and contributor context Nusrat, Humza, Thind, Kundan · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: aba4c2b1-d1f7-4db8-8dd5-e0ae6ffdb42f. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-14T03:01:55.209Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 28694a7dadc132a8914e43275fc23fb81d8379e69ad37ab36483d41d2e39b839. Source text hash: 58b45b43c7010a9dfd3935ebda2c5ff25abc0d5cb17e4835e1aa7a23e24351fd. ## Revision and source-check history ### Version 1 — 2026-11-14T03:01:55.209Z Humza Nusrat and Kundan Thind present OpenGray, a benchmark designed to test whether radiotherapy planning agents make appropriate decisions, not just whether their plans achieve favorable dose results. It adds tasks with known correct decisions to the OpenKBP-Opt head-and-neck dataset, including cases where dose goals cannot all be met and a feasible control with a stored plan meeting every hard goal. Across 30 patients, systems using Terra had high decision accuracy overall, but escalated only some contradictions that required arithmetic on the anatomy. All four systems submitted the feasible control, yet 24% of those plans missed a hard goal met by the stored plan. In separate stereotactic radiosurgery runs, SAGE identified a goal that failed at every attempt, but had no action for stopping. The authors argue that aggregate scores can conceal important decision failures. Their findings suggest that evaluating dose alone may miss whether an agent recognizes an infeasible task, requests review, or stops. The reported results come from the benchmark and runs described in the paper; they do not establish how these agents would perform in broader clinical use. How well would decision-focused evaluation predict safe behavior in clinical settings? ## Original source arXiv cs.AI · Article Original source date: Oct 27, 2026 (2026-10-27). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-14T03:01:55.209Z. First observed: 2026-10-28T05:52:45.035Z. Retrieved: 2026-10-30T06:16:22.820Z. - [OpenGray: a benchmark for evaluating the decisions of radiotherapy planning agents](https://arxiv.org/abs/2610.37081) ## Limits The evaluation covered 30 patients in the benchmark and logs from 39 stereotactic radiosurgery runs; the evidence does not establish performance in broader clinical use. The abstract reports selected results but does not provide enough detail to assess how the findings generalize beyond the evaluated tasks and systems. ## Open question How well would decision-focused evaluation predict safe behavior in clinical settings? ## Source-stated Nusrat and Thind developed OpenGray to evaluate radiotherapy planning agents on tasks with known correct decisions, in addition to dose. > This study developed OpenGray, a benchmark of planning tasks with known correct decisions. - [Supporting source](https://arxiv.org/abs/2610.37081) ## Source-stated In the study's 30-patient evaluation, systems using Terra achieved high overall decision accuracy but escalated only a portion of overlap contradictions. > decision accuracies of 95.4% and 92.1%, respectively, yet escalated only 50% (95% confidence interval, 25-75%) and 25% (10-40%) of overlap contradictions - [Supporting source](https://arxiv.org/abs/2610.37081) ## Source-stated All four evaluated systems submitted the feasible control, but 24% of those plans missed a hard goal met by the stored plan. > All four systems submitted every feasible control, but 24% of these plans missed a hard goal that the stored plan had met. - [Supporting source](https://arxiv.org/abs/2610.37081) ## Source-stated In four SAGE runs, the agent identified a repeatedly failing goal but had no action for stopping. > In four SAGE runs, one goal failed at every attempt; the agent identified it in every reply but had no action for stopping. - [Supporting source](https://arxiv.org/abs/2610.37081) ## Source and contributor context Nusrat, Humza, Thind, Kundan · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: aba4c2b1-d1f7-4db8-8dd5-e0ae6ffdb42f. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-14T03:01:55.209Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 28694a7dadc132a8914e43275fc23fb81d8379e69ad37ab36483d41d2e39b839. Source text hash: 58b45b43c7010a9dfd3935ebda2c5ff25abc0d5cb17e4835e1aa7a23e24351fd. ### uncertain — 2026-11-14T03:01:55.209Z — anthropic/claude-opus-5.5 Publication: aba4c2b1-d1f7-4db8-8dd5-e0ae6ffdb42f attribution — pass: The draft correctly attributes OpenGray to authors Humza Nusrat and Kundan Thind in an arXiv cs.AI paper. It frames the findings and conclusions as theirs, for example 'The authors argue', rather than as established fact. > Authors:Humza Nusrat, Kundan Thind summary — uncertain: The figures and findings are faithful, but a general reader lacks essential context. The summary never says these agents are LLM-based. It also leaves Terra (a commercial LLM) and SAGE (a clinical LLM agent) unexplained, along with 'feasible control', 'hard goal' and 'escalated'. It drops the authors' explicit recommendation that agents be given an action for stopping and requesting review. Smallest repair: state 'agents based on large language models', gloss Terra and SAGE, and restore the stop-action recommendation. > Radiotherapy planning agents based on large language models (LLMs) decide when to submit a plan or request review but are evaluated mainly by dose. date — pass: The originalPublished date, 2026-10-27, matches the v1 arXiv submission date. The later feed date of 2026-10-29 is correctly not used. > \[Submitted on 27 Oct 2026\] support — pass: Each claim matches the abstract: the decision accuracies, the low escalation rates, the 24% hard-goal misses and the four SAGE runs. The inference about dose-only evaluation follows from the authors' conclusions. The limitations and the question stay appropriately cautious and do not overstate clinical reliability. > Favorable aggregate scores concealed specific decision failures. Planning agents should be evaluated on their decisions as well as their dose media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/a1e392ffda7319c594f29daa139a01c3bb2e9e9f8907e808b858c1afc6fbd2db/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-nusrat-and-thind-introduce-opengray-a-radiotherapy-a1e392ff.json)