{"id":"d32a9f42dff07eb6175e09013e0369cd0d4391b05c71e7d88ef2296e4e159e23","title":"arXiv: When AI Analysts Disagree: Agentic Multiverse Analysis and the Robustness of Empirical Research","argument":"Authors Ruijiang Gao and Steven Chong Xiao argue that agentic multiverse analysis (AMA) can make hidden analytical choices visible and help human experts assess how robust empirical findings are. They call those choices “forks”: defensible ways an analyst might conduct an analysis. In a pilot using one dataset and one set of research questions, 150 AI coding agents disagreed through methodological choices also made by human analysts, though across a narrower range. AI peer review barely changed their disagreement; exposure to highly rated example reports sharply reduced it, largely through imitation rather than analysis. The authors’ framework has agents map a paper’s possible forks, assess them against a validity rubric, run the validated specifications, and check one another’s work under separated duties. Across ten studies, it reproduced the original estimates and found that some conclusions held across all validated specifications, while others changed with individual choices the papers did not discuss. The authors suggest these assessments could act as robustness proxies. They report that the work takes hours of agent time and tens of dollars per paper, and say AI makes multiverse analysis practical where few research teams could previously afford it.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The pilot uses one dataset and one set of research questions; its findings do not establish how AI agents would disagree across other settings. The AMA case studies cover ten empirical studies from one leading management journal’s replication packages; the abstract does not establish how well the results generalize beyond them. The authors describe AMA-based measures as possible robustness proxies, not as established measures of real-world reliability.","speaker":{"name":"Gao, Ruijiang, Xiao, Steven Chong","roleAtSource":"Article author"},"source":{"title":"When AI Analysts Disagree: Agentic Multiverse Analysis and the Robustness of Empirical Research","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.25198","publishedAt":"2026-11-18","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors propose AMA as a way to expose how defensible analytical choices affect empirical conclusions, for human experts’ decision support.","excerpt":"We develop an agentic multiverse analysis (AMA) framework that makes a conclusion's sensitivity to those choices visible at scale, and we position it as a decision-support instrument for human experts.","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"},{"text":"In a pilot involving one dataset and one set of research questions, the AI agents disagreed over defensible methodological choices, but across a narrower range than human analysts.","excerpt":"A pilot study first deploys 150 autonomous AI coding agents on one dataset and one set of research questions and finds that they disagree through the same defensible methodological choices human analysts make, though over a narrower range.","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"},{"text":"The pilot found that AI peer review scarcely changed disagreement, while exposure to highly rated example reports reduced it largely through imitation.","excerpt":"AI peer review leaves their dispersion almost unchanged. Exposure to top-rated exemplar reports then collapses it, largely through imitation rather than analysis.","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"},{"text":"AMA maps possible analytical choices, assesses them, runs validated specifications, and has agents cross-examine one another under separated duties.","excerpt":"AMA has agents enumerate a published paper's possible fork space, score every fork against a validity rubric, execute the validated specification grid, and cross-examine one another's work under strict separation of duties.","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"},{"text":"Across ten studies, the framework reproduced the original estimates; some conclusions held across all validated specifications, while others were sensitive to individual choices not discussed in the papers.","excerpt":"Applied to ten empirical studies from a leading management journal's replication packages (1,755 specification estimates), the framework reproduces every paper's original estimates, and shows that some conclusions hold across every specification the assessment validates, while others are sensitive","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"},{"text":"The authors suggest AMA-based measures could serve as proxies for the robustness of empirical findings.","excerpt":"The case studies suggest our AMA-based measures can serve as robustness proxies for empirical findings.","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"},{"text":"The authors report that their exercises take hours of agent time and tens of dollars per paper.","excerpt":"our exercises take hours of agent time and tens of dollars per paper.","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.25198","checkedAt":"2026-11-20T23:20:09.515Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well would AMA’s robustness assessments hold up when applied to empirical studies beyond the ten in these case studies?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv: When AI Analysts Disagree: Agentic Multiverse Analysis and the Robustness of Empirical Research","summary":"Authors Ruijiang Gao and Steven Chong Xiao argue that agentic multiverse analysis (AMA) can make hidden analytical choices visible and help human experts assess how robust empirical findings are. They call those choices “forks”: defensible ways an analyst might conduct an analysis. In a pilot using one dataset and one set of research questions, 150 AI coding agents disagreed through methodological choices also made by human analysts, though across a narrower range. AI peer review barely changed their disagreement; exposure to highly rated example reports sharply reduced it, largely through imitation rather than analysis. The authors’ framework has agents map a paper’s possible forks, assess them against a validity rubric, run the validated specifications, and check one another’s work under separated duties. Across ten studies, it reproduced the original estimates and found that some conclusions held across all validated specifications, while others changed with individual choices the papers did not discuss. The authors suggest these assessments could act as robustness proxies. They report that the work takes hours of agent time and tens of dollars per paper, and say AI makes multiverse analysis practical where few research teams could previously afford it.","question":"How well would AMA’s robustness assessments hold up when applied to empirical studies beyond the ten in these case studies?","claims":[{"text":"The authors propose AMA as a way to expose how defensible analytical choices affect empirical conclusions, for human experts’ decision support.","excerpt":"We develop an agentic multiverse analysis (AMA) framework that makes a conclusion's sensitivity to those choices visible at scale, and we position it as a decision-support instrument for human experts.","support":"source-stated"},{"text":"In a pilot involving one dataset and one set of research questions, the AI agents disagreed over defensible methodological choices, but across a narrower range than human analysts.","excerpt":"A pilot study first deploys 150 autonomous AI coding agents on one dataset and one set of research questions and finds that they disagree through the same defensible methodological choices human analysts make, though over a narrower range.","support":"source-stated"},{"text":"The pilot found that AI peer review scarcely changed disagreement, while exposure to highly rated example reports reduced it largely through imitation.","excerpt":"AI peer review leaves their dispersion almost unchanged. Exposure to top-rated exemplar reports then collapses it, largely through imitation rather than analysis.","support":"source-stated"},{"text":"AMA maps possible analytical choices, assesses them, runs validated specifications, and has agents cross-examine one another under separated duties.","excerpt":"AMA has agents enumerate a published paper's possible fork space, score every fork against a validity rubric, execute the validated specification grid, and cross-examine one another's work under strict separation of duties.","support":"source-stated"},{"text":"Across ten studies, the framework reproduced the original estimates; some conclusions held across all validated specifications, while others were sensitive to individual choices not discussed in the papers.","excerpt":"Applied to ten empirical studies from a leading management journal's replication packages (1,755 specification estimates), the framework reproduces every paper's original estimates, and shows that some conclusions hold across every specification the assessment validates, while others are sensitive","support":"source-stated"},{"text":"The authors suggest AMA-based measures could serve as proxies for the robustness of empirical findings.","excerpt":"The case studies suggest our AMA-based measures can serve as robustness proxies for empirical findings.","support":"source-stated"},{"text":"The authors report that their exercises take hours of agent time and tens of dollars per paper.","excerpt":"our exercises take hours of agent time and tens of dollars per paper.","support":"source-stated"}],"limitations":["The pilot uses one dataset and one set of research questions; its findings do not establish how AI agents would disagree across other settings.","The AMA case studies cover ten empirical studies from one leading management journal’s replication packages; the abstract does not establish how well the results generalize beyond them.","The authors describe AMA-based measures as possible robustness proxies, not as established measures of real-world reliability."],"mediaId":"18b6ec3bfe3653cd04da591879427c6af21ab8824d23194bac5b5a0f61fa399d","correctionReason":null,"id":"d32a9f42dff07eb6175e09013e0369cd0d4391b05c71e7d88ef2296e4e159e23","publicationId":"52dbae03-1899-44c4-8b8f-05d52d6d0149","version":1,"sourceUrl":"https://arxiv.org/abs/2611.25198","sourceTitle":"When AI Analysts Disagree: Agentic Multiverse Analysis and the Robustness of Empirical Research","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-18","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-20T05:01:57.867Z","createdAt":"2026-11-20T06:16:39.097Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"18b6ec3bfe3653cd04da591879427c6af21ab8824d23194bac5b5a0f61fa399d","kind":"article","url":"https://arxiv.org/abs/2611.25198","sourceUrl":"https://arxiv.org/abs/2611.25198","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-20T06:15:51.743Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.25198","checkedAt":"2026-11-20T23:20:09.515Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"18b6ec3bfe3653cd04da591879427c6af21ab8824d23194bac5b5a0f61fa399d","confidence":0.73,"provider":"typesafe","model":"jev-1.13.0","inputHash":"095643e797f2f1fc75da040aade39d58925bc833a1ec96a39fee39885d11b0dc","decidedAt":"2026-11-20T06:15:52.471Z"},"provenance":{"textHash":"decebb36513cbe26b734c274df8c1212db7acee176071d5ecd5cc5cd0bea8fff","observationId":"892195c2a69a6c9da479ef1035a890cc253b0634f230998ca1de91a4bd8fe5a2","retrievedAt":"2026-11-20T06:15:51.610Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"3ada73cd-97c1-4dd9-8422-8782714cc277"},"reviewer":{"model":"gpt-6-luna","runId":"17f473ee-9db1-4c4c-8876-64804dd84804"}},"audit":{"status":"passed","reviewedAt":"2026-11-20T06:16:39.097Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Gao, Ruijiang","url":null,"kind":"unknown"},{"name":"Xiao, Steven Chong","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Gao, Ruijiang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Gao, Ruijiang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.25198","locator":"head/meta[name=author|citation_author]"},"assertionId":"01a8c6676d5f1e9791434874a3a00bbf87d2771cac2aa14901c5156a337c5e2f:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-20T23:20:09.515Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"01a8c6676d5f1e9791434874a3a00bbf87d2771cac2aa14901c5156a337c5e2f","sourceUrl":"https://arxiv.org/abs/2611.25198","contentSha256":"b46ced55b804bba3e3caf9d456e739a1f9702bc34420618757dd3fe9aee52f44","textHash":"decebb36513cbe26b734c274df8c1212db7acee176071d5ecd5cc5cd0bea8fff","observedAt":"2026-11-20T23:20:09.515Z"}},{"actor":{"name":"Xiao, Steven Chong","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Xiao, Steven Chong","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.25198","locator":"head/meta[name=author|citation_author]"},"assertionId":"01a8c6676d5f1e9791434874a3a00bbf87d2771cac2aa14901c5156a337c5e2f:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-20T23:20:09.515Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"01a8c6676d5f1e9791434874a3a00bbf87d2771cac2aa14901c5156a337c5e2f","sourceUrl":"https://arxiv.org/abs/2611.25198","contentSha256":"b46ced55b804bba3e3caf9d456e739a1f9702bc34420618757dd3fe9aee52f44","textHash":"decebb36513cbe26b734c274df8c1212db7acee176071d5ecd5cc5cd0bea8fff","observedAt":"2026-11-20T23:20:09.515Z"}}],"participationCoverage":{"total":2,"returned":2,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.25198","snapshotId":"01a8c6676d5f1e9791434874a3a00bbf87d2771cac2aa14901c5156a337c5e2f","contentSha256":"b46ced55b804bba3e3caf9d456e739a1f9702bc34420618757dd3fe9aee52f44","textHash":"decebb36513cbe26b734c274df8c1212db7acee176071d5ecd5cc5cd0bea8fff","observedAt":"2026-11-20T23:20:09.515Z","basis":"publisher-page-metadata","sourceLinks":40,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/d32a9f42dff07eb6175e09013e0369cd0d4391b05c71e7d88ef2296e4e159e23/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-when-ai-analysts-disagree-agentic-multiverse-analysis-and-d32a9f42","formats":{"html":"/readings/arxiv-when-ai-analysts-disagree-agentic-multiverse-analysis-and-d32a9f42","markdown":"/readings/arxiv-when-ai-analysts-disagree-agentic-multiverse-analysis-and-d32a9f42.md","json":"/readings/arxiv-when-ai-analysts-disagree-agentic-multiverse-analysis-and-d32a9f42.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-agentic","name":"Agentic AI","explanation":"AI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer.","referenceUrl":"https://developers.google.com/machine-learning/glossary#agentic"},{"id":"term-ai-agent","name":"AI agent","explanation":"An AI system that chooses and carries out actions toward a goal, often using software tools.","referenceUrl":"https://www.ibm.com/think/topics/ai-agents"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"},{"id":"time","name":"TIME","kind":"organization","context":"TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.","referenceUrls":["https://time.com/"],"path":"/entities/time","json":"/entities/time.json","markdown":"/entities/time.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"d32a9f42dff07eb6175e09013e0369cd0d4391b05c71e7d88ef2296e4e159e23","revision":"52dbae03-1899-44c4-8b8f-05d52d6d0149","revisionKind":"publication-id","title":"arXiv: When AI Analysts Disagree: Agentic Multiverse Analysis and the Robustness of Empirical Research","url":"https://escapemeridian.com/readings/arxiv-when-ai-analysts-disagree-agentic-multiverse-analysis-and-d32a9f42"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.25198","revision":"decebb36513cbe26b734c274df8c1212db7acee176071d5ecd5cc5cd0bea8fff","revisionKind":"sha256-source-text","title":"When AI Analysts Disagree: Agentic Multiverse Analysis and the Robustness of Empirical Research","url":"https://arxiv.org/abs/2611.25198"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter."},"revisions":[{"id":"d32a9f42dff07eb6175e09013e0369cd0d4391b05c71e7d88ef2296e4e159e23","title":"arXiv: When AI Analysts Disagree: Agentic Multiverse Analysis and the Robustness of Empirical Research","argument":"Authors Ruijiang Gao and Steven Chong Xiao argue that agentic multiverse analysis (AMA) can make hidden analytical choices visible and help human experts assess how robust empirical findings are. They call those choices “forks”: defensible ways an analyst might conduct an analysis. In a pilot using one dataset and one set of research questions, 150 AI coding agents disagreed through methodological choices also made by human analysts, though across a narrower range. AI peer review barely changed their disagreement; exposure to highly rated example reports sharply reduced it, largely through imitation rather than analysis. The authors’ framework has agents map a paper’s possible forks, assess them against a validity rubric, run the validated specifications, and check one another’s work under separated duties. Across ten studies, it reproduced the original estimates and found that some conclusions held across all validated specifications, while others changed with individual choices the papers did not discuss. The authors suggest these assessments could act as robustness proxies. They report that the work takes hours of agent time and tens of dollars per paper, and say AI makes multiverse analysis practical where few research teams could previously afford it.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The pilot uses one dataset and one set of research questions; its findings do not establish how AI agents would disagree across other settings. The AMA case studies cover ten empirical studies from one leading management journal’s replication packages; the abstract does not establish how well the results generalize beyond them. The authors describe AMA-based measures as possible robustness proxies, not as established measures of real-world reliability.","speaker":{"name":"Gao, Ruijiang, Xiao, Steven Chong","roleAtSource":"Article author"},"source":{"title":"When AI Analysts Disagree: Agentic Multiverse Analysis and the Robustness of Empirical Research","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.25198","publishedAt":"2026-11-18","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors propose AMA as a way to expose how defensible analytical choices affect empirical conclusions, for human experts’ decision support.","excerpt":"We develop an agentic multiverse analysis (AMA) framework that makes a conclusion's sensitivity to those choices visible at scale, and we position it as a decision-support instrument for human experts.","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"},{"text":"In a pilot involving one dataset and one set of research questions, the AI agents disagreed over defensible methodological choices, but across a narrower range than human analysts.","excerpt":"A pilot study first deploys 150 autonomous AI coding agents on one dataset and one set of research questions and finds that they disagree through the same defensible methodological choices human analysts make, though over a narrower range.","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"},{"text":"The pilot found that AI peer review scarcely changed disagreement, while exposure to highly rated example reports reduced it largely through imitation.","excerpt":"AI peer review leaves their dispersion almost unchanged. Exposure to top-rated exemplar reports then collapses it, largely through imitation rather than analysis.","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"},{"text":"AMA maps possible analytical choices, assesses them, runs validated specifications, and has agents cross-examine one another under separated duties.","excerpt":"AMA has agents enumerate a published paper's possible fork space, score every fork against a validity rubric, execute the validated specification grid, and cross-examine one another's work under strict separation of duties.","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"},{"text":"Across ten studies, the framework reproduced the original estimates; some conclusions held across all validated specifications, while others were sensitive to individual choices not discussed in the papers.","excerpt":"Applied to ten empirical studies from a leading management journal's replication packages (1,755 specification estimates), the framework reproduces every paper's original estimates, and shows that some conclusions hold across every specification the assessment validates, while others are sensitive","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"},{"text":"The authors suggest AMA-based measures could serve as proxies for the robustness of empirical findings.","excerpt":"The case studies suggest our AMA-based measures can serve as robustness proxies for empirical findings.","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"},{"text":"The authors report that their exercises take hours of agent time and tens of dollars per paper.","excerpt":"our exercises take hours of agent time and tens of dollars per paper.","url":"https://arxiv.org/abs/2611.25198","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.25198","checkedAt":"2026-11-20T23:20:09.515Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well would AMA’s robustness assessments hold up when applied to empirical studies beyond the ten in these case studies?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv: When AI Analysts Disagree: Agentic Multiverse Analysis and the Robustness of Empirical Research","summary":"Authors Ruijiang Gao and Steven Chong Xiao argue that agentic multiverse analysis (AMA) can make hidden analytical choices visible and help human experts assess how robust empirical findings are. They call those choices “forks”: defensible ways an analyst might conduct an analysis. In a pilot using one dataset and one set of research questions, 150 AI coding agents disagreed through methodological choices also made by human analysts, though across a narrower range. AI peer review barely changed their disagreement; exposure to highly rated example reports sharply reduced it, largely through imitation rather than analysis. The authors’ framework has agents map a paper’s possible forks, assess them against a validity rubric, run the validated specifications, and check one another’s work under separated duties. Across ten studies, it reproduced the original estimates and found that some conclusions held across all validated specifications, while others changed with individual choices the papers did not discuss. The authors suggest these assessments could act as robustness proxies. They report that the work takes hours of agent time and tens of dollars per paper, and say AI makes multiverse analysis practical where few research teams could previously afford it.","question":"How well would AMA’s robustness assessments hold up when applied to empirical studies beyond the ten in these case studies?","claims":[{"text":"The authors propose AMA as a way to expose how defensible analytical choices affect empirical conclusions, for human experts’ decision support.","excerpt":"We develop an agentic multiverse analysis (AMA) framework that makes a conclusion's sensitivity to those choices visible at scale, and we position it as a decision-support instrument for human experts.","support":"source-stated"},{"text":"In a pilot involving one dataset and one set of research questions, the AI agents disagreed over defensible methodological choices, but across a narrower range than human analysts.","excerpt":"A pilot study first deploys 150 autonomous AI coding agents on one dataset and one set of research questions and finds that they disagree through the same defensible methodological choices human analysts make, though over a narrower range.","support":"source-stated"},{"text":"The pilot found that AI peer review scarcely changed disagreement, while exposure to highly rated example reports reduced it largely through imitation.","excerpt":"AI peer review leaves their dispersion almost unchanged. Exposure to top-rated exemplar reports then collapses it, largely through imitation rather than analysis.","support":"source-stated"},{"text":"AMA maps possible analytical choices, assesses them, runs validated specifications, and has agents cross-examine one another under separated duties.","excerpt":"AMA has agents enumerate a published paper's possible fork space, score every fork against a validity rubric, execute the validated specification grid, and cross-examine one another's work under strict separation of duties.","support":"source-stated"},{"text":"Across ten studies, the framework reproduced the original estimates; some conclusions held across all validated specifications, while others were sensitive to individual choices not discussed in the papers.","excerpt":"Applied to ten empirical studies from a leading management journal's replication packages (1,755 specification estimates), the framework reproduces every paper's original estimates, and shows that some conclusions hold across every specification the assessment validates, while others are sensitive","support":"source-stated"},{"text":"The authors suggest AMA-based measures could serve as proxies for the robustness of empirical findings.","excerpt":"The case studies suggest our AMA-based measures can serve as robustness proxies for empirical findings.","support":"source-stated"},{"text":"The authors report that their exercises take hours of agent time and tens of dollars per paper.","excerpt":"our exercises take hours of agent time and tens of dollars per paper.","support":"source-stated"}],"limitations":["The pilot uses one dataset and one set of research questions; its findings do not establish how AI agents would disagree across other settings.","The AMA case studies cover ten empirical studies from one leading management journal’s replication packages; the abstract does not establish how well the results generalize beyond them.","The authors describe AMA-based measures as possible robustness proxies, not as established measures of real-world reliability."],"mediaId":"18b6ec3bfe3653cd04da591879427c6af21ab8824d23194bac5b5a0f61fa399d","correctionReason":null,"id":"d32a9f42dff07eb6175e09013e0369cd0d4391b05c71e7d88ef2296e4e159e23","publicationId":"52dbae03-1899-44c4-8b8f-05d52d6d0149","version":1,"sourceUrl":"https://arxiv.org/abs/2611.25198","sourceTitle":"When AI Analysts Disagree: Agentic Multiverse Analysis and the Robustness of Empirical Research","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-18","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-20T05:01:57.867Z","createdAt":"2026-11-20T06:16:39.097Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"18b6ec3bfe3653cd04da591879427c6af21ab8824d23194bac5b5a0f61fa399d","kind":"article","url":"https://arxiv.org/abs/2611.25198","sourceUrl":"https://arxiv.org/abs/2611.25198","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-20T06:15:51.743Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.25198","checkedAt":"2026-11-20T23:20:09.515Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"18b6ec3bfe3653cd04da591879427c6af21ab8824d23194bac5b5a0f61fa399d","confidence":0.73,"provider":"typesafe","model":"jev-1.13.0","inputHash":"095643e797f2f1fc75da040aade39d58925bc833a1ec96a39fee39885d11b0dc","decidedAt":"2026-11-20T06:15:52.471Z"},"provenance":{"textHash":"decebb36513cbe26b734c274df8c1212db7acee176071d5ecd5cc5cd0bea8fff","observationId":"892195c2a69a6c9da479ef1035a890cc253b0634f230998ca1de91a4bd8fe5a2","retrievedAt":"2026-11-20T06:15:51.610Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"3ada73cd-97c1-4dd9-8422-8782714cc277"},"reviewer":{"model":"gpt-6-luna","runId":"17f473ee-9db1-4c4c-8876-64804dd84804"}},"audit":{"status":"passed","reviewedAt":"2026-11-20T06:16:39.097Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Gao, Ruijiang","url":null,"kind":"unknown"},{"name":"Xiao, Steven Chong","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Gao, Ruijiang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Gao, Ruijiang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.25198","locator":"head/meta[name=author|citation_author]"},"assertionId":"01a8c6676d5f1e9791434874a3a00bbf87d2771cac2aa14901c5156a337c5e2f:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-20T23:20:09.515Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"01a8c6676d5f1e9791434874a3a00bbf87d2771cac2aa14901c5156a337c5e2f","sourceUrl":"https://arxiv.org/abs/2611.25198","contentSha256":"b46ced55b804bba3e3caf9d456e739a1f9702bc34420618757dd3fe9aee52f44","textHash":"decebb36513cbe26b734c274df8c1212db7acee176071d5ecd5cc5cd0bea8fff","observedAt":"2026-11-20T23:20:09.515Z"}},{"actor":{"name":"Xiao, Steven Chong","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Xiao, Steven Chong","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.25198","locator":"head/meta[name=author|citation_author]"},"assertionId":"01a8c6676d5f1e9791434874a3a00bbf87d2771cac2aa14901c5156a337c5e2f:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-20T23:20:09.515Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"01a8c6676d5f1e9791434874a3a00bbf87d2771cac2aa14901c5156a337c5e2f","sourceUrl":"https://arxiv.org/abs/2611.25198","contentSha256":"b46ced55b804bba3e3caf9d456e739a1f9702bc34420618757dd3fe9aee52f44","textHash":"decebb36513cbe26b734c274df8c1212db7acee176071d5ecd5cc5cd0bea8fff","observedAt":"2026-11-20T23:20:09.515Z"}}],"participationCoverage":{"total":2,"returned":2,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.25198","snapshotId":"01a8c6676d5f1e9791434874a3a00bbf87d2771cac2aa14901c5156a337c5e2f","contentSha256":"b46ced55b804bba3e3caf9d456e739a1f9702bc34420618757dd3fe9aee52f44","textHash":"decebb36513cbe26b734c274df8c1212db7acee176071d5ecd5cc5cd0bea8fff","observedAt":"2026-11-20T23:20:09.515Z","basis":"publisher-page-metadata","sourceLinks":40,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/d32a9f42dff07eb6175e09013e0369cd0d4391b05c71e7d88ef2296e4e159e23/source"}}}],"audits":[{"publicationId":"52dbae03-1899-44c4-8b8f-05d52d6d0149","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading correctly credits Ruijiang Gao and Steven Chong Xiao. It frames the framework, findings, proxy suggestion and cost figures as the authors' claims and reports, not as independently established facts.","excerpt":"Authors:Ruijiang Gao, Steven Chong Xiao"},{"dimension":"summary","status":"pass","explanation":"The summary faithfully follows the abstract: forks, the 150-agent pilot, peer review versus exemplar imitation, the AMA steps, the ten-study results and the cost claim. Forks are defined in plain language, and AMA's purpose is clear enough for a general reader. The question about generalization follows from the evidence. One small optional addition would be the authors' stated implications for replication policy and AI-assisted peer review; the reading does not need it.","excerpt":"Every empirical analysis on observational data rests on dozens of defensible analytical choices (\"forks\") that its readers cannot see."},{"dimension":"date","status":"pass","explanation":"The displayed originalPublished date of 2026-11-18 matches this arXiv entry's v1 submission date. The feed date of Nov 20 is later. The note that this paper subsumes an earlier arXiv ID does not change the first publication date of this entry.","excerpt":"[Submitted on 18 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim is supported by a verbatim abstract excerpt, and wording such as 'sharply reduced' fairly renders 'collapses'. The limitations correctly confine the pilot to one dataset and the case studies to one journal's replication packages. They also keep robustness proxies as a suggestion rather than proven reliability. The reading invents no beneficiaries, costs or forecasts.","excerpt":"The case studies suggest our AMA-based measures can serve as robustness proxies for empirical findings."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified, direct arXiv abstract page for this paper. It is the most direct way to access the source.","excerpt":null}],"createdAt":"2026-11-20T06:16:39.097Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}