arXiv cs.AI: Nusrat and Thind introduce OpenGray, a radiotherapy planning benchmark

arXiv cs.AI: Nusrat and Thind introduce OpenGray, a radiotherapy planning benchmark

Humza Nusrat and Kundan Thind present OpenGray, a benchmark designed to test whether radiotherapy planning agents make appropriate decisions, not just whether their plans achieve favorable dose results. It adds tasks with known correct decisions to the OpenKBP-Opt head-and-neck dataset, including cases where dose goals cannot all be met and a feasible control with a stored plan meeting every hard goal. Across 30 patients, systems using Terra had high decision accuracy overall, but escalated only some contradictions that required arithmetic on the anatomy. All four systems submitted the feasible control, yet 24% of those plans missed a hard goal met by the stored plan. In separate stereotactic radiosurgery runs, SAGE identified a goal that failed at every attempt, but had no action for stopping. The authors argue that aggregate scores can conceal important decision failures. Their findings suggest that evaluating dose alone may miss whether an agent recognizes an infeasible task, requests review, or stops. The reported results come from the benchmark and runs described in the paper; they do not establish how these agents would perform in broader clinical use. How well would decision-focused evaluation predict safe behavior in clinical settings?

Publication: aba4c2b1-d1f7-4db8-8dd5-e0ae6ffdb42f

attribution — pass: The draft correctly attributes OpenGray to authors Humza Nusrat and Kundan Thind in an arXiv cs.AI paper. It frames the findings and conclusions as theirs, for example 'The authors argue', rather than as established fact.

Authors:Humza Nusrat, Kundan Thind

summary — uncertain: The figures and findings are faithful, but a general reader lacks essential context. The summary never says these agents are LLM-based. It also leaves Terra (a commercial LLM) and SAGE (a clinical LLM agent) unexplained, along with 'feasible control', 'hard goal' and 'escalated'. It drops the authors' explicit recommendation that agents be given an action for stopping and requesting review. Smallest repair: state 'agents based on large language models', gloss Terra and SAGE, and restore the stop-action recommendation.

Radiotherapy planning agents based on large language models (LLMs) decide when to submit a plan or request review but are evaluated mainly by dose.

date — pass: The originalPublished date, 2026-10-27, matches the v1 arXiv submission date. The later feed date of 2026-10-29 is correctly not used.

[Submitted on 27 Oct 2026]

support — pass: Each claim matches the abstract: the decision accuracies, the low escalation rates, the 24% hard-goal misses and the four SAGE runs. The inference about dose-only evaluation follows from the authors' conclusions. The limitations and the question stay appropriately cautious and do not overstate clinical reliability.

Favorable aggregate scores concealed specific decision failures. Planning agents should be evaluated on their decisions as well as their dose

media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source.