{"id":"57301541c6689454c33fb3f3e198219c31be0ac843c4bf0d755dff885d38666f","title":"Delin Wang and coauthors introduce RubricAtlas for evaluating deep research agents","argument":"The authors introduce RubricAtlas, a way to evaluate deep research agents by checking whether reports cover recurring analytical questions for a research type, rather than only whether they contain relevant facts. They derive requirements from expert reports and link binary checkpoints to report passages, assessing completeness separately from argument validity. The authors report that choosing requirements from different reference reports changes evaluation scores and agent rankings. Their approach aims to make comparisons traceable to evidence, while avoiding a fixed answer to each analytical question. The study is built from Chinese financial research reports; whether the method works for other research domains or practical evaluation settings remains open.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evaluation is based on Chinese financial research reports; the evidence does not establish how well the method transfers to other research domains. The abstract reports controlled experiments and judge-agreement measures, but does not establish performance in practical deployments or whether the rubrics capture every important analytical requirement.","speaker":{"name":"Wang, Delin, Wang, Sinuo, Sun, Rui, Luan, Beidi, Shi, Qiming, Li, Jing, Jiang, Daxin, Bai, Zuo","roleAtSource":"Article author"},"source":{"title":"RubricAtlas: Mining Type-Level Rubrics from Expert Reports to Evaluate Deep Research Agents","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.36878","publishedAt":"2026-10-27","dateBasis":"Date basis not provided."},"evidence":[{"text":"RubricAtlas derives shared analytical requirements from expert reports on different objects within the same research type, without prescribing answers.","excerpt":"The requirements give related tasks a shared standard for what to analyse, without prescribing their answers.","url":"https://arxiv.org/abs/2610.36878","relationship":"Source-stated"},{"text":"The framework connects binary checkpoints to report passages and evaluates completeness separately from argument validity.","excerpt":"Our framework links binary checkpoints to report passages and evaluates completeness separately from argument validity.","url":"https://arxiv.org/abs/2610.36878","relationship":"Source-stated"},{"text":"In the authors’ reference-report selection experiments, individual reports covered about half of their type’s analytical questions, and selecting requirements from different reports changed scores and agent rankings.","excerpt":"individual reports cover only about half of their type's analytical questions, and that selecting requirements from different reports changes evaluation scores and agent rankings.","url":"https://arxiv.org/abs/2610.36878","relationship":"Source-stated"},{"text":"The authors say the framework was built from Chinese financial research reports and covers 99 research types.","excerpt":"Built from 3,079 Chinese financial research reports, RubricAtlas covers 99 research types.","url":"https://arxiv.org/abs/2610.36878","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36878","checkedAt":"2026-11-06T15:04:06.784Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Would type-level rubrics derived from expert reports produce stable evaluations in research domains beyond Chinese financial reports?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Delin Wang and coauthors introduce RubricAtlas for evaluating deep research agents","summary":"The authors introduce RubricAtlas, a way to evaluate deep research agents by checking whether reports cover recurring analytical questions for a research type, rather than only whether they contain relevant facts. They derive requirements from expert reports and link binary checkpoints to report passages, assessing completeness separately from argument validity. The authors report that choosing requirements from different reference reports changes evaluation scores and agent rankings. Their approach aims to make comparisons traceable to evidence, while avoiding a fixed answer to each analytical question. The study is built from Chinese financial research reports; whether the method works for other research domains or practical evaluation settings remains open.","question":"Would type-level rubrics derived from expert reports produce stable evaluations in research domains beyond Chinese financial reports?","claims":[{"text":"RubricAtlas derives shared analytical requirements from expert reports on different objects within the same research type, without prescribing answers.","excerpt":"The requirements give related tasks a shared standard for what to analyse, without prescribing their answers.","support":"source-stated"},{"text":"The framework connects binary checkpoints to report passages and evaluates completeness separately from argument validity.","excerpt":"Our framework links binary checkpoints to report passages and evaluates completeness separately from argument validity.","support":"source-stated"},{"text":"In the authors’ reference-report selection experiments, individual reports covered about half of their type’s analytical questions, and selecting requirements from different reports changed scores and agent rankings.","excerpt":"individual reports cover only about half of their type's analytical questions, and that selecting requirements from different reports changes evaluation scores and agent rankings.","support":"source-stated"},{"text":"The authors say the framework was built from Chinese financial research reports and covers 99 research types.","excerpt":"Built from 3,079 Chinese financial research reports, RubricAtlas covers 99 research types.","support":"source-stated"}],"limitations":["The evaluation is based on Chinese financial research reports; the evidence does not establish how well the method transfers to other research domains.","The abstract reports controlled experiments and judge-agreement measures, but does not establish performance in practical deployments or whether the rubrics capture every important analytical requirement."],"mediaId":"e37e7605cf26a455c4c0352020c25e399d06840e9c9e042cc0277e314a58e382","correctionReason":null,"id":"57301541c6689454c33fb3f3e198219c31be0ac843c4bf0d755dff885d38666f","publicationId":"978221e5-07f7-4cb8-882b-2684bde947a9","version":1,"sourceUrl":"https://arxiv.org/abs/2610.36878","sourceTitle":"RubricAtlas: Mining Type-Level Rubrics from Expert Reports to Evaluate Deep Research Agents","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-27","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-28T08:24:36.300Z","createdAt":"2026-11-06T10:49:18.977Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"e37e7605cf26a455c4c0352020c25e399d06840e9c9e042cc0277e314a58e382","kind":"article","url":"https://arxiv.org/abs/2610.36878","sourceUrl":"https://arxiv.org/abs/2610.36878","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-30T00:46:40.129Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36878","checkedAt":"2026-11-06T15:04:06.784Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"e37e7605cf26a455c4c0352020c25e399d06840e9c9e042cc0277e314a58e382","confidence":0.64,"provider":"typesafe","model":"jev-1.13.0","inputHash":"c58902fa79c29d05753df19dced871b1d417c1798198c9b5c4e48b144840127e","decidedAt":"2026-10-30T00:46:40.691Z","redecided":{"from":"stream-admission-v5","under":"stream-admission-v5+confidence-0.5"}},"provenance":{"textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observationId":"dc1f5d7717277586dc3790106662bb285f931929e93b9bfc924f1dc4bf2c495d","retrievedAt":"2026-10-30T00:46:40.053Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"1fba7716-0968-4f61-8805-150e3fc58098"},"reviewer":{"model":"gpt-6-luna","runId":"9466e8bf-2943-4e34-843a-e680b2eae233"}},"audit":{"status":"passed","reviewedAt":"2026-11-06T10:49:18.977Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Wang, Delin","url":null,"kind":"unknown"},{"name":"Wang, Sinuo","url":null,"kind":"unknown"},{"name":"Sun, Rui","url":null,"kind":"unknown"},{"name":"Luan, Beidi","url":null,"kind":"unknown"},{"name":"Shi, Qiming","url":null,"kind":"unknown"},{"name":"Li, Jing","url":null,"kind":"unknown"},{"name":"Jiang, Daxin","url":null,"kind":"unknown"},{"name":"Bai, Zuo","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Wang, Delin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wang, Delin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Wang, Sinuo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wang, Sinuo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Sun, Rui","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sun, Rui","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Luan, Beidi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Luan, Beidi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Shi, Qiming","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Shi, Qiming","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Li, Jing","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Li, Jing","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Jiang, Daxin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jiang, Daxin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Bai, Zuo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Bai, Zuo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}}],"participationCoverage":{"total":8,"returned":8,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.36878","snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z","basis":"publisher-page-metadata","sourceLinks":47,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/57301541c6689454c33fb3f3e198219c31be0ac843c4bf0d755dff885d38666f/source"}},"canonicalUrl":"https://escapemeridian.com/readings/delin-wang-and-coauthors-introduce-rubricatlas-for-evaluating-57301541","formats":{"html":"/readings/delin-wang-and-coauthors-introduce-rubricatlas-for-evaluating-57301541","markdown":"/readings/delin-wang-and-coauthors-introduce-rubricatlas-for-evaluating-57301541.md","json":"/readings/delin-wang-and-coauthors-introduce-rubricatlas-for-evaluating-57301541.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"57301541c6689454c33fb3f3e198219c31be0ac843c4bf0d755dff885d38666f","revision":"978221e5-07f7-4cb8-882b-2684bde947a9","revisionKind":"publication-id","title":"Delin Wang and coauthors introduce RubricAtlas for evaluating deep research agents","url":"https://escapemeridian.com/readings/delin-wang-and-coauthors-introduce-rubricatlas-for-evaluating-57301541"},"source":{"type":"source","id":"https://arxiv.org/abs/2610.36878","revision":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","revisionKind":"sha256-source-text","title":"RubricAtlas: Mining Type-Level Rubrics from Expert Reports to Evaluate Deep Research Agents","url":"https://arxiv.org/abs/2610.36878"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"57301541c6689454c33fb3f3e198219c31be0ac843c4bf0d755dff885d38666f","title":"Delin Wang and coauthors introduce RubricAtlas for evaluating deep research agents","argument":"The authors introduce RubricAtlas, a way to evaluate deep research agents by checking whether reports cover recurring analytical questions for a research type, rather than only whether they contain relevant facts. They derive requirements from expert reports and link binary checkpoints to report passages, assessing completeness separately from argument validity. The authors report that choosing requirements from different reference reports changes evaluation scores and agent rankings. Their approach aims to make comparisons traceable to evidence, while avoiding a fixed answer to each analytical question. The study is built from Chinese financial research reports; whether the method works for other research domains or practical evaluation settings remains open.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evaluation is based on Chinese financial research reports; the evidence does not establish how well the method transfers to other research domains. The abstract reports controlled experiments and judge-agreement measures, but does not establish performance in practical deployments or whether the rubrics capture every important analytical requirement.","speaker":{"name":"Wang, Delin, Wang, Sinuo, Sun, Rui, Luan, Beidi, Shi, Qiming, Li, Jing, Jiang, Daxin, Bai, Zuo","roleAtSource":"Article author"},"source":{"title":"RubricAtlas: Mining Type-Level Rubrics from Expert Reports to Evaluate Deep Research Agents","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.36878","publishedAt":"2026-10-27","dateBasis":"Date basis not provided."},"evidence":[{"text":"RubricAtlas derives shared analytical requirements from expert reports on different objects within the same research type, without prescribing answers.","excerpt":"The requirements give related tasks a shared standard for what to analyse, without prescribing their answers.","url":"https://arxiv.org/abs/2610.36878","relationship":"Source-stated"},{"text":"The framework connects binary checkpoints to report passages and evaluates completeness separately from argument validity.","excerpt":"Our framework links binary checkpoints to report passages and evaluates completeness separately from argument validity.","url":"https://arxiv.org/abs/2610.36878","relationship":"Source-stated"},{"text":"In the authors’ reference-report selection experiments, individual reports covered about half of their type’s analytical questions, and selecting requirements from different reports changed scores and agent rankings.","excerpt":"individual reports cover only about half of their type's analytical questions, and that selecting requirements from different reports changes evaluation scores and agent rankings.","url":"https://arxiv.org/abs/2610.36878","relationship":"Source-stated"},{"text":"The authors say the framework was built from Chinese financial research reports and covers 99 research types.","excerpt":"Built from 3,079 Chinese financial research reports, RubricAtlas covers 99 research types.","url":"https://arxiv.org/abs/2610.36878","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36878","checkedAt":"2026-11-06T15:04:06.784Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Would type-level rubrics derived from expert reports produce stable evaluations in research domains beyond Chinese financial reports?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Delin Wang and coauthors introduce RubricAtlas for evaluating deep research agents","summary":"The authors introduce RubricAtlas, a way to evaluate deep research agents by checking whether reports cover recurring analytical questions for a research type, rather than only whether they contain relevant facts. They derive requirements from expert reports and link binary checkpoints to report passages, assessing completeness separately from argument validity. The authors report that choosing requirements from different reference reports changes evaluation scores and agent rankings. Their approach aims to make comparisons traceable to evidence, while avoiding a fixed answer to each analytical question. The study is built from Chinese financial research reports; whether the method works for other research domains or practical evaluation settings remains open.","question":"Would type-level rubrics derived from expert reports produce stable evaluations in research domains beyond Chinese financial reports?","claims":[{"text":"RubricAtlas derives shared analytical requirements from expert reports on different objects within the same research type, without prescribing answers.","excerpt":"The requirements give related tasks a shared standard for what to analyse, without prescribing their answers.","support":"source-stated"},{"text":"The framework connects binary checkpoints to report passages and evaluates completeness separately from argument validity.","excerpt":"Our framework links binary checkpoints to report passages and evaluates completeness separately from argument validity.","support":"source-stated"},{"text":"In the authors’ reference-report selection experiments, individual reports covered about half of their type’s analytical questions, and selecting requirements from different reports changed scores and agent rankings.","excerpt":"individual reports cover only about half of their type's analytical questions, and that selecting requirements from different reports changes evaluation scores and agent rankings.","support":"source-stated"},{"text":"The authors say the framework was built from Chinese financial research reports and covers 99 research types.","excerpt":"Built from 3,079 Chinese financial research reports, RubricAtlas covers 99 research types.","support":"source-stated"}],"limitations":["The evaluation is based on Chinese financial research reports; the evidence does not establish how well the method transfers to other research domains.","The abstract reports controlled experiments and judge-agreement measures, but does not establish performance in practical deployments or whether the rubrics capture every important analytical requirement."],"mediaId":"e37e7605cf26a455c4c0352020c25e399d06840e9c9e042cc0277e314a58e382","correctionReason":null,"id":"57301541c6689454c33fb3f3e198219c31be0ac843c4bf0d755dff885d38666f","publicationId":"978221e5-07f7-4cb8-882b-2684bde947a9","version":1,"sourceUrl":"https://arxiv.org/abs/2610.36878","sourceTitle":"RubricAtlas: Mining Type-Level Rubrics from Expert Reports to Evaluate Deep Research Agents","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-27","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-28T08:24:36.300Z","createdAt":"2026-11-06T10:49:18.977Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"e37e7605cf26a455c4c0352020c25e399d06840e9c9e042cc0277e314a58e382","kind":"article","url":"https://arxiv.org/abs/2610.36878","sourceUrl":"https://arxiv.org/abs/2610.36878","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-30T00:46:40.129Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36878","checkedAt":"2026-11-06T15:04:06.784Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"e37e7605cf26a455c4c0352020c25e399d06840e9c9e042cc0277e314a58e382","confidence":0.64,"provider":"typesafe","model":"jev-1.13.0","inputHash":"c58902fa79c29d05753df19dced871b1d417c1798198c9b5c4e48b144840127e","decidedAt":"2026-10-30T00:46:40.691Z","redecided":{"from":"stream-admission-v5","under":"stream-admission-v5+confidence-0.5"}},"provenance":{"textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observationId":"dc1f5d7717277586dc3790106662bb285f931929e93b9bfc924f1dc4bf2c495d","retrievedAt":"2026-10-30T00:46:40.053Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"1fba7716-0968-4f61-8805-150e3fc58098"},"reviewer":{"model":"gpt-6-luna","runId":"9466e8bf-2943-4e34-843a-e680b2eae233"}},"audit":{"status":"passed","reviewedAt":"2026-11-06T10:49:18.977Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Wang, Delin","url":null,"kind":"unknown"},{"name":"Wang, Sinuo","url":null,"kind":"unknown"},{"name":"Sun, Rui","url":null,"kind":"unknown"},{"name":"Luan, Beidi","url":null,"kind":"unknown"},{"name":"Shi, Qiming","url":null,"kind":"unknown"},{"name":"Li, Jing","url":null,"kind":"unknown"},{"name":"Jiang, Daxin","url":null,"kind":"unknown"},{"name":"Bai, Zuo","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Wang, Delin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wang, Delin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Wang, Sinuo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wang, Sinuo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Sun, Rui","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sun, Rui","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Luan, Beidi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Luan, Beidi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Shi, Qiming","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Shi, Qiming","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Li, Jing","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Li, Jing","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Jiang, Daxin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jiang, Daxin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}},{"actor":{"name":"Bai, Zuo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Bai, Zuo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36878","locator":"head/meta[name=author|citation_author]"},"assertionId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-06T11:20:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","sourceUrl":"https://arxiv.org/abs/2610.36878","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z"}}],"participationCoverage":{"total":8,"returned":8,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.36878","snapshotId":"ebaa8d6d59e2ef8641219ce1a6937852e51d499c466fb7e84e16ab4027854a6b","contentSha256":"9876be507fcc372606ca44299c8dda80798484b76b087dc70f38fa5d98c9f339","textHash":"d1dbe5d65e82f08291c68d11d45aea5a00ecce5df5ed921d3cd631f265d5e7f0","observedAt":"2026-11-06T11:20:07.139Z","basis":"publisher-page-metadata","sourceLinks":47,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/57301541c6689454c33fb3f3e198219c31be0ac843c4bf0d755dff885d38666f/source"}}}],"audits":[{"publicationId":"978221e5-07f7-4cb8-882b-2684bde947a9","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The title credits Delin Wang and coauthors, matching the arXiv author list. The summary frames the findings as the authors' reports rather than established facts.","excerpt":"Authors:Delin Wang, Sinuo Wang, Rui Sun, Beidi Luan, Qiming Shi, Jing Li, Daxin Jiang, Zuo Bai"},{"dimension":"summary","status":"pass","explanation":"The summary faithfully conveys the main idea. RubricAtlas uses type-level analytical requirements drawn from expert reports, and it judges completeness separately from argument validity. It also reports the key finding that the choice of reference report shifts scores and rankings. The reports context lets a general reader infer what deep research agents do. A tooltip for the term would still help. The summary omits the judge-agreement and padding/deletion results, but this does not distort meaning. The question follows from the stated scope.","excerpt":"Deep research agents conduct multi-step Web searches and synthesise retrieved information into analytical reports."},{"dimension":"date","status":"pass","explanation":"The originalPublished value of 2026-10-27 matches the v1 submission date on arXiv. It is not the later feed date.","excerpt":"[Submitted on 27 Oct 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim matches the abstract, and the excerpts are verbatim. The limitations are appropriately conditional: they note the Chinese financial data scope and the lack of deployment evidence without overstating anything. No benefits or costs are invented.","excerpt":"individual reports cover only about half of their type's analytical questions, and that selecting requirements from different reports changes evaluation scores and agent rankings."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper. It is the most direct way to reach the source.","excerpt":null}],"createdAt":"2026-11-06T10:49:18.977Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}