Kim, Changyeon, Nah, Yekyung, Lee, Kimin · Nov 4, 2026
RACE: Calibrated Cumulative Scoring for Rollout-Free Robot Policy Checkpoint Selection
Markdown · JSON · Original source
Changyeon Kim, Yekyung Nah and Kimin Lee propose RACE, a rollout-free metric for ranking robot policy checkpoints without running costly real-world trials for each candidate. They argue that averaged validation loss can rank checkpoints inconsistently with deployment performance: it penalizes action deviations without accounting for recovery, and weights later reference predictions equally even when earlier deviations may have taken execution away from the reference. RACE instead compares action deviations with state-conditional thresholds derived from similar states in held-out successful trajectories. It combines scores over using a running product, reducing the contribution of later reference predictions after a substantial deviation, then averages cumulative scores across time and trajectories. The authors report that RACE outperforms baselines in simulation across validation sources, while several baselines degrade sharply when the validation set changes. On real robots, they report that RACE’s rankings align with deployment success across evaluated policy backbones and tasks of varying difficulty. They also report that perturbation analyses distinguish deviations with markedly different effects on task success, even when baselines respond similarly. The proposed approach could help researchers select checkpoints with fewer costly trials, but the reported results do 3
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 4, 2026 (2026-11-04).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-09T08:26:08.003Z. First observed: 2026-11-05T05:06:44.342Z. Retrieved: 2026-11-09T08:25:07.821Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The supplied evidence is the paper’s abstract and does not give experimental details such as task counts or quantitative results. The reported real-robot ranking alignment applies to the evaluated policy backbones and tasks; the abstract does not establish performance beyond those settings.
Open question
How would RACE’s checkpoint rankings compare on tasks and policy backbones beyond those evaluated?
Source-stated
The authors propose RACE as a rollout-free metric to rank robot policy checkpoints using held-out successful trajectories.
We propose RACE, a novel rollout-free metric for ranking checkpoints using a fixed set of held-out successful trajectories.
Source-stated
The authors identify two limitations of averaged validation loss: it does not account for recovery from action deviations, and it gives later reference predictions equal weight after earlier deviations.
it penalizes action deviations without considering whether the policy can recover from them, and gives later reference predictions equal weight even when earlier deviations may lead execution away from the reference.
Source-stated
RACE calibrates action deviations with state-conditional thresholds and aggregates scores using a running product.
RACE calibrates action deviations against state-conditional thresholds derived from action differences at similar states in these trajectories. We aggregate these scores through a running product
Source-stated
The authors report that RACE outperforms baselines in simulation across validation sources, while several baselines degrade sharply when the validation set changes.
In simulation, RACE outperforms baselines across validation sources, while several baselines degrade sharply when the validation set changes.
Source-stated
The authors report that RACE’s rankings align with deployment success on the real robots, policy backbones and tasks they evaluated.
On real robots, its rankings align with deployment success across the evaluated policy backbones and tasks of varying difficulty.
Source-stated
The authors report that perturbation analyses found RACE distinguished deviations with markedly different effects on task success, even when baselines responded similarly.
Perturbation analyses further show that RACE distinguishes deviations with markedly different effects on task success, even when baselines respond similarly.
Escape Meridian interpretation
If the reported ranking alignment generalizes, RACE could help researchers select checkpoints while reducing the need for costly real-world trials; the abstract does not establish this benefit beyond the evaluated settings.
directly measuring task success requires costly real-world trials for each candidate checkpoint.
Source and contributor context
Kim, Changyeon, Nah, Yekyung, Lee, Kimin · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: 16c0645d-20dc-437e-8930-313fb1dea79b.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T08:26:08.003Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: d10a893416f6d39cb7c55f26296135269f25b4c9c992e467dc3ad66d80486cd3. Source text hash: c68fab92e0dfb251198227aae19e07c94a5590a2d83a1990963244a76334377a.
Revision and source-check history
Version 1 · 2026-11-09T08:26:08.003Z
RACE: Calibrated Cumulative Scoring for Rollout-Free Robot Policy Checkpoint Selection
Changyeon Kim, Yekyung Nah and Kimin Lee propose RACE, a rollout-free metric for ranking robot policy checkpoints without running costly real-world trials for each candidate. They argue that averaged validation loss can rank checkpoints inconsistently with deployment performance: it penalizes action deviations without accounting for recovery, and weights later reference predictions equally even when earlier deviations may have taken execution away from the reference. RACE instead compares action deviations with state-conditional thresholds derived from similar states in held-out successful trajectories. It combines scores over using a running product, reducing the contribution of later reference predictions after a substantial deviation, then averages cumulative scores across time and trajectories. The authors report that RACE outperforms baselines in simulation across validation sources, while several baselines degrade sharply when the validation set changes. On real robots, they report that RACE’s rankings align with deployment success across evaluated policy backbones and tasks of varying difficulty. They also report that perturbation analyses distinguish deviations with markedly different effects on task success, even when baselines respond similarly. The proposed approach could help researchers select checkpoints with fewer costly trials, but the reported results do 3
Original source
cs.AI · Article
Source details
Original source date: Nov 4, 2026 (2026-11-04).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-09T08:26:08.003Z. First observed: 2026-11-05T05:06:44.342Z. Retrieved: 2026-11-09T08:25:07.821Z.
Limits
The supplied evidence is the paper’s abstract and does not give experimental details such as task counts or quantitative results. The reported real-robot ranking alignment applies to the evaluated policy backbones and tasks; the abstract does not establish performance beyond those settings.
Open question
How would RACE’s checkpoint rankings compare on tasks and policy backbones beyond those evaluated?
Source-stated
The authors propose RACE as a rollout-free metric to rank robot policy checkpoints using held-out successful trajectories.
We propose RACE, a novel rollout-free metric for ranking checkpoints using a fixed set of held-out successful trajectories.
Source-stated
The authors identify two limitations of averaged validation loss: it does not account for recovery from action deviations, and it gives later reference predictions equal weight after earlier deviations.
it penalizes action deviations without considering whether the policy can recover from them, and gives later reference predictions equal weight even when earlier deviations may lead execution away from the reference.
Source-stated
RACE calibrates action deviations with state-conditional thresholds and aggregates scores using a running product.
RACE calibrates action deviations against state-conditional thresholds derived from action differences at similar states in these trajectories. We aggregate these scores through a running product
Source-stated
The authors report that RACE outperforms baselines in simulation across validation sources, while several baselines degrade sharply when the validation set changes.
In simulation, RACE outperforms baselines across validation sources, while several baselines degrade sharply when the validation set changes.
Source-stated
The authors report that RACE’s rankings align with deployment success on the real robots, policy backbones and tasks they evaluated.
On real robots, its rankings align with deployment success across the evaluated policy backbones and tasks of varying difficulty.
Source-stated
The authors report that perturbation analyses found RACE distinguished deviations with markedly different effects on task success, even when baselines responded similarly.
Perturbation analyses further show that RACE distinguishes deviations with markedly different effects on task success, even when baselines respond similarly.
Escape Meridian interpretation
If the reported ranking alignment generalizes, RACE could help researchers select checkpoints while reducing the need for costly real-world trials; the abstract does not establish this benefit beyond the evaluated settings.
directly measuring task success requires costly real-world trials for each candidate checkpoint.
Source and contributor context
Kim, Changyeon, Nah, Yekyung, Lee, Kimin · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: 16c0645d-20dc-437e-8930-313fb1dea79b.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T08:26:08.003Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: d10a893416f6d39cb7c55f26296135269f25b4c9c992e467dc3ad66d80486cd3. Source text hash: c68fab92e0dfb251198227aae19e07c94a5590a2d83a1990963244a76334377a.
uncertain · 2026-11-09T08:26:08.003Z · anthropic/claude-opus-5.5
Publication: 16c0645d-20dc-437e-8930-313fb1dea79b
attribution — pass: The draft correctly credits Changyeon Kim, Yekyung Nah and Kimin Lee as the authors of the arXiv paper. It frames the results as what the authors report rather than as established fact.
Authors:Changyeon Kim, Yekyung Nah, Kimin Lee
summary — uncertain: The summary ends mid-sentence with a garbled fragment: 'but the reported results do 3'. This cuts off the hedge that should limit the benefit claim. Smallest repair: complete the sentence, for example by noting that the abstract does not establish the benefit beyond the evaluated settings. Several central terms are also unexplained for non-specialists: checkpoint, rollout-free, validation loss, policy backbones and trajectories. Short plain-language glosses or term-library tooltips are needed so readers can follow what RACE ranks and why.
Assessing the deployment performance of learned robot policies during training is challenging
date — pass: The original publication date of 2026-11-04 matches the v1 submission date on arXiv. The feed date of Nov 5 is later and was correctly not used.
[Submitted on 4 Nov 2026]
support — pass: Every source-stated claim matches the abstract and keeps its scope, including the evaluated backbones and tasks and the different validation sources. The interpretive claim about reducing costly trials is explicitly conditional and grounded in the paper's stated motivation. However, its hedge is truncated in the summary text, which is noted under summary. The question does not presuppose an answer.
On real robots, its rankings align with deployment success across the evaluated policy backbones and tasks of varying difficulty.
media — pass: The selected media is the verified direct link to the original arXiv abstract page, which is the source itself. No better source media was found.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv