{"id":"b6041664238d9d075e8a2d15508d1493c8e2a8e4f2b17043cac204b5a008ee8a","title":"arXiv cs.AI: CuraEval tests whether curated robot demonstrations beat simple baselines","argument":"Authors Jiuyi Xu, Meida Chen, Yang Ye and Yangming Shi introduce CuraEval, a framework for comparing ways to select robot demonstrations—recordings of operators performing tasks. They argue that differing selection and evaluation protocols make it hard to tell whether curation beats simple baselines, so CuraEval compares fixed size-matched random subsets and full-data training on mixed-skill robomimic data. In their reported tests, the PSD score largely tracks episode length on square, and selecting the shortest half of demonstrations improves square success over random selection with Diffusion Policy. But results vary with task and evaluation protocol. The adapted geometric-entropy LOO selector performs worse than random selection in reported comparisons, and changing checkpoint reporting, rollout count and evaluation seed jointly reverses the direction of a length-based selection result against full-data training. After Holm correction, no evaluated curated condition significantly outperforms full-data training, though the authors say confidence intervals still permit useful gains. Escape Meridian interpretation: These results make curation a question of careful comparison, not a demonstrated general advantage; the abstract reports specific tasks, methods and conditions, rather than establishing how the findings transfer beyond them.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evidence provided is the paper’s abstract, which reports results on mixed-skill robomimic data and named tasks; it does not establish how the results transfer beyond the tested conditions. The abstract says checkpoint reporting, rollout count and evaluation seed changed jointly, so it does not isolate which change accounts for the altered comparison.","speaker":{"name":"Xu, Jiuyi, Chen, Meida, Ye, Yang, Shi, Yangming","roleAtSource":"Article author"},"source":{"title":"CuraEval: Evaluating Demonstration Curation on Mixed-Skill Robot Data","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.38221","publishedAt":"2026-10-28","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors introduce CuraEval to compare demonstration-selection methods against size-matched random subsets and full-data training.","excerpt":"We introduce \\textbf{CuraEval}, a framework combining mathematical analysis of demonstration scores with controlled comparisons against fixed size-matched random subsets and full-data training","url":"https://arxiv.org/abs/2610.38221","relationship":"Source-stated"},{"text":"On square, the evaluated PSD score produces nearly the same ranking as episode length.","excerpt":"produces nearly the same ranking as episode length on \\textit{square} ($\\rho=0.999$).","url":"https://arxiv.org/abs/2610.38221","relationship":"Source-stated"},{"text":"With Diffusion Policy, selecting the shortest half of demonstrations raises square success compared with random selection in the reported test.","excerpt":"selecting the shortest 50\\% of demonstrations improves \\textit{square} success from 70.7\\% under random selection to 76.5\\%.","url":"https://arxiv.org/abs/2610.38221","relationship":"Source-stated"},{"text":"The adapted geometric-entropy LOO selector performs worse than random selection in the reported comparisons on can and square.","excerpt":"reduces success at 25\\% retention from 94.7\\% to 74.0\\% on \\textit{can} and achieves 41.7\\% on \\textit{square}, below the mean success rates of all three independently sampled random subsets (59.8--62.5\\%).","url":"https://arxiv.org/abs/2610.38221","relationship":"Source-stated"},{"text":"Changing checkpoint reporting, rollout count and evaluation seed jointly changes the reported comparison between length-based selection and full-data training on square.","excerpt":"improves success over full-data training from 76.7\\% to 86.7\\% under the original protocol, but lowers it from 80.5\\% to 79.5\\% after checkpoint reporting, rollout count, and evaluation seed change jointly.","url":"https://arxiv.org/abs/2610.38221","relationship":"Source-stated"},{"text":"The authors report that no evaluated curated condition significantly outperforms full-data training after Holm correction, while confidence intervals still permit useful gains.","excerpt":"No evaluated curated condition significantly outperforms full-data training after Holm correction, although confidence intervals still permit useful gains.","url":"https://arxiv.org/abs/2610.38221","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.38221","checkedAt":"2026-11-12T05:44:07.583Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Would CuraEval’s comparisons show a significant advantage for any curated condition under other tasks or evaluation protocols?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: CuraEval tests whether curated robot demonstrations beat simple baselines","summary":"Authors Jiuyi Xu, Meida Chen, Yang Ye and Yangming Shi introduce CuraEval, a framework for comparing ways to select robot demonstrations—recordings of operators performing tasks. They argue that differing selection and evaluation protocols make it hard to tell whether curation beats simple baselines, so CuraEval compares fixed size-matched random subsets and full-data training on mixed-skill robomimic data. In their reported tests, the PSD score largely tracks episode length on square, and selecting the shortest half of demonstrations improves square success over random selection with Diffusion Policy. But results vary with task and evaluation protocol. The adapted geometric-entropy LOO selector performs worse than random selection in reported comparisons, and changing checkpoint reporting, rollout count and evaluation seed jointly reverses the direction of a length-based selection result against full-data training. After Holm correction, no evaluated curated condition significantly outperforms full-data training, though the authors say confidence intervals still permit useful gains. Escape Meridian interpretation: These results make curation a question of careful comparison, not a demonstrated general advantage; the abstract reports specific tasks, methods and conditions, rather than establishing how the findings transfer beyond them.","question":"Would CuraEval’s comparisons show a significant advantage for any curated condition under other tasks or evaluation protocols?","claims":[{"text":"The authors introduce CuraEval to compare demonstration-selection methods against size-matched random subsets and full-data training.","excerpt":"We introduce \\textbf{CuraEval}, a framework combining mathematical analysis of demonstration scores with controlled comparisons against fixed size-matched random subsets and full-data training","support":"source-stated"},{"text":"On square, the evaluated PSD score produces nearly the same ranking as episode length.","excerpt":"produces nearly the same ranking as episode length on \\textit{square} ($\\rho=0.999$).","support":"source-stated"},{"text":"With Diffusion Policy, selecting the shortest half of demonstrations raises square success compared with random selection in the reported test.","excerpt":"selecting the shortest 50\\% of demonstrations improves \\textit{square} success from 70.7\\% under random selection to 76.5\\%.","support":"source-stated"},{"text":"The adapted geometric-entropy LOO selector performs worse than random selection in the reported comparisons on can and square.","excerpt":"reduces success at 25\\% retention from 94.7\\% to 74.0\\% on \\textit{can} and achieves 41.7\\% on \\textit{square}, below the mean success rates of all three independently sampled random subsets (59.8--62.5\\%).","support":"source-stated"},{"text":"Changing checkpoint reporting, rollout count and evaluation seed jointly changes the reported comparison between length-based selection and full-data training on square.","excerpt":"improves success over full-data training from 76.7\\% to 86.7\\% under the original protocol, but lowers it from 80.5\\% to 79.5\\% after checkpoint reporting, rollout count, and evaluation seed change jointly.","support":"source-stated"},{"text":"The authors report that no evaluated curated condition significantly outperforms full-data training after Holm correction, while confidence intervals still permit useful gains.","excerpt":"No evaluated curated condition significantly outperforms full-data training after Holm correction, although confidence intervals still permit useful gains.","support":"source-stated"}],"limitations":["The evidence provided is the paper’s abstract, which reports results on mixed-skill robomimic data and named tasks; it does not establish how the results transfer beyond the tested conditions.","The abstract says checkpoint reporting, rollout count and evaluation seed changed jointly, so it does not isolate which change accounts for the altered comparison."],"mediaId":"f4c9accbbdb40bcf55cf57b23c2f456dee6ba033ccbe9ea928e8fbf15dfc4151","correctionReason":null,"id":"b6041664238d9d075e8a2d15508d1493c8e2a8e4f2b17043cac204b5a008ee8a","publicationId":"3ae0d1e8-d852-4e5a-86cb-e6c70689ff93","version":1,"sourceUrl":"https://arxiv.org/abs/2610.38221","sourceTitle":"CuraEval: Evaluating Demonstration Curation on Mixed-Skill Robot Data","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-28","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-30T06:45:28.187Z","createdAt":"2026-11-11T12:41:00.634Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"f4c9accbbdb40bcf55cf57b23c2f456dee6ba033ccbe9ea928e8fbf15dfc4151","kind":"article","url":"https://arxiv.org/abs/2610.38221","sourceUrl":"https://arxiv.org/abs/2610.38221","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-31T09:32:19.365Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.38221","checkedAt":"2026-11-12T05:44:07.583Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"f4c9accbbdb40bcf55cf57b23c2f456dee6ba033ccbe9ea928e8fbf15dfc4151","confidence":0.7100000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"e51506687acd4f5bcd838154ab82474b150e401514576bdcd588681c8297476e","decidedAt":"2026-10-31T09:32:19.930Z","redecided":{"from":"stream-admission-v5","under":"stream-admission-v5+confidence-0.5"}},"provenance":{"textHash":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","observationId":"2de6b130e1bcad1be90ef80e177e84c7cb0901bee367dc50d0eaf590a2f8fe7f","retrievedAt":"2026-10-31T09:32:19.281Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"3008e2bd-daa9-433e-8ae0-5eb14be23453"},"reviewer":{"model":"gpt-6-luna","runId":"f7eea87e-c6af-407e-8c62-d5d1c278ce7b"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-11T12:41:00.634Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Xu, Jiuyi","url":null,"kind":"unknown"},{"name":"Chen, Meida","url":null,"kind":"unknown"},{"name":"Ye, Yang","url":null,"kind":"unknown"},{"name":"Shi, Yangming","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Xu, Jiuyi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Xu, Jiuyi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38221","locator":"head/meta[name=author|citation_author]"},"assertionId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T05:44:07.583Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea","sourceUrl":"https://arxiv.org/abs/2610.38221","contentSha256":"b5e8832ed684594bc7fab8c3b6e09f39d82cdb6f827143c9d09dbc91053c37a1","textHash":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","observedAt":"2026-11-12T05:44:07.583Z"}},{"actor":{"name":"Chen, Meida","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chen, Meida","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38221","locator":"head/meta[name=author|citation_author]"},"assertionId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T05:44:07.583Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea","sourceUrl":"https://arxiv.org/abs/2610.38221","contentSha256":"b5e8832ed684594bc7fab8c3b6e09f39d82cdb6f827143c9d09dbc91053c37a1","textHash":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","observedAt":"2026-11-12T05:44:07.583Z"}},{"actor":{"name":"Ye, Yang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ye, Yang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38221","locator":"head/meta[name=author|citation_author]"},"assertionId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T05:44:07.583Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea","sourceUrl":"https://arxiv.org/abs/2610.38221","contentSha256":"b5e8832ed684594bc7fab8c3b6e09f39d82cdb6f827143c9d09dbc91053c37a1","textHash":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","observedAt":"2026-11-12T05:44:07.583Z"}},{"actor":{"name":"Shi, Yangming","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Shi, Yangming","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38221","locator":"head/meta[name=author|citation_author]"},"assertionId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T05:44:07.583Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea","sourceUrl":"https://arxiv.org/abs/2610.38221","contentSha256":"b5e8832ed684594bc7fab8c3b6e09f39d82cdb6f827143c9d09dbc91053c37a1","textHash":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","observedAt":"2026-11-12T05:44:07.583Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.38221","snapshotId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea","contentSha256":"b5e8832ed684594bc7fab8c3b6e09f39d82cdb6f827143c9d09dbc91053c37a1","textHash":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","observedAt":"2026-11-12T05:44:07.583Z","basis":"publisher-page-metadata","sourceLinks":44,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/b6041664238d9d075e8a2d15508d1493c8e2a8e4f2b17043cac204b5a008ee8a/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-curaeval-tests-whether-curated-robot-demonstrations-b6041664","formats":{"html":"/readings/arxiv-cs-ai-curaeval-tests-whether-curated-robot-demonstrations-b6041664","markdown":"/readings/arxiv-cs-ai-curaeval-tests-whether-curated-robot-demonstrations-b6041664.md","json":"/readings/arxiv-cs-ai-curaeval-tests-whether-curated-robot-demonstrations-b6041664.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-confidence-interval","name":"Confidence interval","explanation":"A range of values consistent with the data at a stated level of confidence.","referenceUrl":"https://en.wikipedia.org/wiki/Confidence_interval"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"b6041664238d9d075e8a2d15508d1493c8e2a8e4f2b17043cac204b5a008ee8a","revision":"3ae0d1e8-d852-4e5a-86cb-e6c70689ff93","revisionKind":"publication-id","title":"arXiv cs.AI: CuraEval tests whether curated robot demonstrations beat simple baselines","url":"https://escapemeridian.com/readings/arxiv-cs-ai-curaeval-tests-whether-curated-robot-demonstrations-b6041664"},"source":{"type":"source","id":"https://arxiv.org/abs/2610.38221","revision":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","revisionKind":"sha256-source-text","title":"CuraEval: Evaluating Demonstration Curation on Mixed-Skill Robot Data","url":"https://arxiv.org/abs/2610.38221"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"b6041664238d9d075e8a2d15508d1493c8e2a8e4f2b17043cac204b5a008ee8a","title":"arXiv cs.AI: CuraEval tests whether curated robot demonstrations beat simple baselines","argument":"Authors Jiuyi Xu, Meida Chen, Yang Ye and Yangming Shi introduce CuraEval, a framework for comparing ways to select robot demonstrations—recordings of operators performing tasks. They argue that differing selection and evaluation protocols make it hard to tell whether curation beats simple baselines, so CuraEval compares fixed size-matched random subsets and full-data training on mixed-skill robomimic data. In their reported tests, the PSD score largely tracks episode length on square, and selecting the shortest half of demonstrations improves square success over random selection with Diffusion Policy. But results vary with task and evaluation protocol. The adapted geometric-entropy LOO selector performs worse than random selection in reported comparisons, and changing checkpoint reporting, rollout count and evaluation seed jointly reverses the direction of a length-based selection result against full-data training. After Holm correction, no evaluated curated condition significantly outperforms full-data training, though the authors say confidence intervals still permit useful gains. Escape Meridian interpretation: These results make curation a question of careful comparison, not a demonstrated general advantage; the abstract reports specific tasks, methods and conditions, rather than establishing how the findings transfer beyond them.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evidence provided is the paper’s abstract, which reports results on mixed-skill robomimic data and named tasks; it does not establish how the results transfer beyond the tested conditions. The abstract says checkpoint reporting, rollout count and evaluation seed changed jointly, so it does not isolate which change accounts for the altered comparison.","speaker":{"name":"Xu, Jiuyi, Chen, Meida, Ye, Yang, Shi, Yangming","roleAtSource":"Article author"},"source":{"title":"CuraEval: Evaluating Demonstration Curation on Mixed-Skill Robot Data","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.38221","publishedAt":"2026-10-28","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors introduce CuraEval to compare demonstration-selection methods against size-matched random subsets and full-data training.","excerpt":"We introduce \\textbf{CuraEval}, a framework combining mathematical analysis of demonstration scores with controlled comparisons against fixed size-matched random subsets and full-data training","url":"https://arxiv.org/abs/2610.38221","relationship":"Source-stated"},{"text":"On square, the evaluated PSD score produces nearly the same ranking as episode length.","excerpt":"produces nearly the same ranking as episode length on \\textit{square} ($\\rho=0.999$).","url":"https://arxiv.org/abs/2610.38221","relationship":"Source-stated"},{"text":"With Diffusion Policy, selecting the shortest half of demonstrations raises square success compared with random selection in the reported test.","excerpt":"selecting the shortest 50\\% of demonstrations improves \\textit{square} success from 70.7\\% under random selection to 76.5\\%.","url":"https://arxiv.org/abs/2610.38221","relationship":"Source-stated"},{"text":"The adapted geometric-entropy LOO selector performs worse than random selection in the reported comparisons on can and square.","excerpt":"reduces success at 25\\% retention from 94.7\\% to 74.0\\% on \\textit{can} and achieves 41.7\\% on \\textit{square}, below the mean success rates of all three independently sampled random subsets (59.8--62.5\\%).","url":"https://arxiv.org/abs/2610.38221","relationship":"Source-stated"},{"text":"Changing checkpoint reporting, rollout count and evaluation seed jointly changes the reported comparison between length-based selection and full-data training on square.","excerpt":"improves success over full-data training from 76.7\\% to 86.7\\% under the original protocol, but lowers it from 80.5\\% to 79.5\\% after checkpoint reporting, rollout count, and evaluation seed change jointly.","url":"https://arxiv.org/abs/2610.38221","relationship":"Source-stated"},{"text":"The authors report that no evaluated curated condition significantly outperforms full-data training after Holm correction, while confidence intervals still permit useful gains.","excerpt":"No evaluated curated condition significantly outperforms full-data training after Holm correction, although confidence intervals still permit useful gains.","url":"https://arxiv.org/abs/2610.38221","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.38221","checkedAt":"2026-11-12T05:44:07.583Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Would CuraEval’s comparisons show a significant advantage for any curated condition under other tasks or evaluation protocols?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: CuraEval tests whether curated robot demonstrations beat simple baselines","summary":"Authors Jiuyi Xu, Meida Chen, Yang Ye and Yangming Shi introduce CuraEval, a framework for comparing ways to select robot demonstrations—recordings of operators performing tasks. They argue that differing selection and evaluation protocols make it hard to tell whether curation beats simple baselines, so CuraEval compares fixed size-matched random subsets and full-data training on mixed-skill robomimic data. In their reported tests, the PSD score largely tracks episode length on square, and selecting the shortest half of demonstrations improves square success over random selection with Diffusion Policy. But results vary with task and evaluation protocol. The adapted geometric-entropy LOO selector performs worse than random selection in reported comparisons, and changing checkpoint reporting, rollout count and evaluation seed jointly reverses the direction of a length-based selection result against full-data training. After Holm correction, no evaluated curated condition significantly outperforms full-data training, though the authors say confidence intervals still permit useful gains. Escape Meridian interpretation: These results make curation a question of careful comparison, not a demonstrated general advantage; the abstract reports specific tasks, methods and conditions, rather than establishing how the findings transfer beyond them.","question":"Would CuraEval’s comparisons show a significant advantage for any curated condition under other tasks or evaluation protocols?","claims":[{"text":"The authors introduce CuraEval to compare demonstration-selection methods against size-matched random subsets and full-data training.","excerpt":"We introduce \\textbf{CuraEval}, a framework combining mathematical analysis of demonstration scores with controlled comparisons against fixed size-matched random subsets and full-data training","support":"source-stated"},{"text":"On square, the evaluated PSD score produces nearly the same ranking as episode length.","excerpt":"produces nearly the same ranking as episode length on \\textit{square} ($\\rho=0.999$).","support":"source-stated"},{"text":"With Diffusion Policy, selecting the shortest half of demonstrations raises square success compared with random selection in the reported test.","excerpt":"selecting the shortest 50\\% of demonstrations improves \\textit{square} success from 70.7\\% under random selection to 76.5\\%.","support":"source-stated"},{"text":"The adapted geometric-entropy LOO selector performs worse than random selection in the reported comparisons on can and square.","excerpt":"reduces success at 25\\% retention from 94.7\\% to 74.0\\% on \\textit{can} and achieves 41.7\\% on \\textit{square}, below the mean success rates of all three independently sampled random subsets (59.8--62.5\\%).","support":"source-stated"},{"text":"Changing checkpoint reporting, rollout count and evaluation seed jointly changes the reported comparison between length-based selection and full-data training on square.","excerpt":"improves success over full-data training from 76.7\\% to 86.7\\% under the original protocol, but lowers it from 80.5\\% to 79.5\\% after checkpoint reporting, rollout count, and evaluation seed change jointly.","support":"source-stated"},{"text":"The authors report that no evaluated curated condition significantly outperforms full-data training after Holm correction, while confidence intervals still permit useful gains.","excerpt":"No evaluated curated condition significantly outperforms full-data training after Holm correction, although confidence intervals still permit useful gains.","support":"source-stated"}],"limitations":["The evidence provided is the paper’s abstract, which reports results on mixed-skill robomimic data and named tasks; it does not establish how the results transfer beyond the tested conditions.","The abstract says checkpoint reporting, rollout count and evaluation seed changed jointly, so it does not isolate which change accounts for the altered comparison."],"mediaId":"f4c9accbbdb40bcf55cf57b23c2f456dee6ba033ccbe9ea928e8fbf15dfc4151","correctionReason":null,"id":"b6041664238d9d075e8a2d15508d1493c8e2a8e4f2b17043cac204b5a008ee8a","publicationId":"3ae0d1e8-d852-4e5a-86cb-e6c70689ff93","version":1,"sourceUrl":"https://arxiv.org/abs/2610.38221","sourceTitle":"CuraEval: Evaluating Demonstration Curation on Mixed-Skill Robot Data","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-28","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-30T06:45:28.187Z","createdAt":"2026-11-11T12:41:00.634Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"f4c9accbbdb40bcf55cf57b23c2f456dee6ba033ccbe9ea928e8fbf15dfc4151","kind":"article","url":"https://arxiv.org/abs/2610.38221","sourceUrl":"https://arxiv.org/abs/2610.38221","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-31T09:32:19.365Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.38221","checkedAt":"2026-11-12T05:44:07.583Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"f4c9accbbdb40bcf55cf57b23c2f456dee6ba033ccbe9ea928e8fbf15dfc4151","confidence":0.7100000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"e51506687acd4f5bcd838154ab82474b150e401514576bdcd588681c8297476e","decidedAt":"2026-10-31T09:32:19.930Z","redecided":{"from":"stream-admission-v5","under":"stream-admission-v5+confidence-0.5"}},"provenance":{"textHash":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","observationId":"2de6b130e1bcad1be90ef80e177e84c7cb0901bee367dc50d0eaf590a2f8fe7f","retrievedAt":"2026-10-31T09:32:19.281Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"3008e2bd-daa9-433e-8ae0-5eb14be23453"},"reviewer":{"model":"gpt-6-luna","runId":"f7eea87e-c6af-407e-8c62-d5d1c278ce7b"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-11T12:41:00.634Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Xu, Jiuyi","url":null,"kind":"unknown"},{"name":"Chen, Meida","url":null,"kind":"unknown"},{"name":"Ye, Yang","url":null,"kind":"unknown"},{"name":"Shi, Yangming","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Xu, Jiuyi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Xu, Jiuyi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38221","locator":"head/meta[name=author|citation_author]"},"assertionId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T05:44:07.583Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea","sourceUrl":"https://arxiv.org/abs/2610.38221","contentSha256":"b5e8832ed684594bc7fab8c3b6e09f39d82cdb6f827143c9d09dbc91053c37a1","textHash":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","observedAt":"2026-11-12T05:44:07.583Z"}},{"actor":{"name":"Chen, Meida","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chen, Meida","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38221","locator":"head/meta[name=author|citation_author]"},"assertionId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T05:44:07.583Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea","sourceUrl":"https://arxiv.org/abs/2610.38221","contentSha256":"b5e8832ed684594bc7fab8c3b6e09f39d82cdb6f827143c9d09dbc91053c37a1","textHash":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","observedAt":"2026-11-12T05:44:07.583Z"}},{"actor":{"name":"Ye, Yang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ye, Yang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38221","locator":"head/meta[name=author|citation_author]"},"assertionId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T05:44:07.583Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea","sourceUrl":"https://arxiv.org/abs/2610.38221","contentSha256":"b5e8832ed684594bc7fab8c3b6e09f39d82cdb6f827143c9d09dbc91053c37a1","textHash":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","observedAt":"2026-11-12T05:44:07.583Z"}},{"actor":{"name":"Shi, Yangming","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Shi, Yangming","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38221","locator":"head/meta[name=author|citation_author]"},"assertionId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T05:44:07.583Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea","sourceUrl":"https://arxiv.org/abs/2610.38221","contentSha256":"b5e8832ed684594bc7fab8c3b6e09f39d82cdb6f827143c9d09dbc91053c37a1","textHash":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","observedAt":"2026-11-12T05:44:07.583Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.38221","snapshotId":"34a2f0766047f1d4b9a7cb07d89062cbb948d9ab4633d1146a6907a8325d20ea","contentSha256":"b5e8832ed684594bc7fab8c3b6e09f39d82cdb6f827143c9d09dbc91053c37a1","textHash":"ecb3ca3c9cefcfa94a0ebac8dc1abcbdbec0071f7d5abba0e5a668cd095eb52a","observedAt":"2026-11-12T05:44:07.583Z","basis":"publisher-page-metadata","sourceLinks":44,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/b6041664238d9d075e8a2d15508d1493c8e2a8e4f2b17043cac204b5a008ee8a/source"}}}],"audits":[{"publicationId":"3ae0d1e8-d852-4e5a-86cb-e6c70689ff93","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The four named authors match the source, and CuraEval and its findings are attributed to them. The 'arXiv cs.AI' label matches the publisher metadata; the paper is primarily cs.RO and cross-listed to cs.AI. The interpretive closing is clearly labeled as Escape Meridian interpretation.","excerpt":"Authors:Jiuyi Xu, Meida Chen, Yang Ye, Yangming Shi"},{"dimension":"summary","status":"uncertain","explanation":"The reading conveys the paper's main point: curation methods do not reliably beat simple baselines, and protocol choices change results. However, 'square' and 'can' appear without any cue that they are robomimic benchmark task names, so 'square success' is opaque to general readers. Smallest repair: say 'the square task' and 'the can task' on first use. PSD score, Diffusion Policy, LOO, Holm correction and episode length also lack plain explanation; tooltip entries should be confirmed. One phrase is also garbled: 'CuraEval compares fixed size-matched random subsets' should say it compares selection methods against those subsets.","excerpt":"controlled comparisons against fixed size-matched random subsets and full-data training on mixed-skill robomimic data"},{"dimension":"date","status":"pass","explanation":"originalPublished 2026-10-28 matches the arXiv v1 submission date. The later feed entry (Oct 31) is correctly not used.","excerpt":"[Submitted on 28 Oct 2026]"},{"dimension":"support","status":"uncertain","explanation":"Each listed claim matches its excerpt, and the Holm-correction caveat and confidence-interval condition are preserved. However, the sentence 'results vary with task and evaluation protocol' goes beyond the abstract. Protocol dependence is shown, but no method is reported to help on one task and hurt on another. The length-based gains are reported only on square, and the geometric-entropy selector underperforms on both tasks. Smallest repair: 'results vary with selection method and evaluation protocol.' The labeled EM interpretation is appropriately hedged and grounded in the Holm result.","excerpt":"No evaluated curated condition significantly outperforms full-data training after Holm correction, although confidence intervals still permit useful gains."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper, which gives readers the easiest access to the source.","excerpt":null}],"createdAt":"2026-11-11T12:41:00.634Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}