{"id":"5c652c1258b9669bdeb39e5a5ec5c425b7175a665c26b8baa588c8e5e238e456","title":"Marginal Revolution introduces a paper arguing many development-economics RCTs are underpowered","argument":"Marginal Revolution introduces a paper by Qingyuan Chai, Kevin Lang and Liang Zhong on the statistical credibility of randomized controlled trials (RCTs) in development economics. The authors report that actual t-statistics peak below the theoretical benchmark expected under 80 percent power, but above values associated with p-hacking; they say this suggests many RCTs are underpowered. Using an empirical Bayes mixture model that accounts for selective publication, they estimate that approximately 30 percent of published rejections are false discoveries, with average directional power of 40 percent. At conventional significance thresholds, they estimate a 66 percent posterior probability of a false discovery. The paper compares two changes: doubling original sample sizes reduces the average false discovery rate by five percentage points, while tightening the significance threshold to 0.005 yields a larger reduction at the cost of total discoveries. It also models replication funding under a fixed budget and evidence standard. Under standard regularity conditions, the authors’ model allocates replications to an intermediate interval of original test statistics, with optimal replication sample size single-peaked in the original t-statistic.","topicIds":["power-and-infrastructure","work-and-agency"],"domain":"work","claimType":"attributed reading","limits":"The analysis described is limited to 745 principal-hypothesis tests from 190 articles in leading economics journals, in development economics. The source presents the replication-policy results as a model under standard regularity conditions; its reported post-replication rates are illustrative calibrations. The source does not specify the evidence standards behind the 1.8–7.8 percent calibration range.","speaker":{"name":"Tyler Cowen","roleAtSource":"Article author"},"source":{"title":"Are RCTs underpowered? - Marginal REVOLUTION","publisher":"Marginal Revolution","format":"Article","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","publishedAt":"2026-11-10T20:05:26+00:00","dateBasis":"Date stated by the publisher."},"evidence":[{"text":"The authors report that t-statistics in their sample peak below the theoretical benchmark expected under 80 percent power, which they say suggests many RCTs are underpowered.","excerpt":"actual t-statistics peak well below the theoretical benchmark of 2.8 expected under 80 percent power, though above values associated with p-hacking. This suggests many RCTs are underpowered.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","relationship":"Source-stated"},{"text":"Using an empirical Bayes mixture model that accounts for selective publication, the authors estimate that approximately 30 percent of published rejections are false discoveries and average directional power is 40 percent.","excerpt":"we find that approximately 30 percent of published rejections are false discoveries, with average directional power at only 40 percent.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","relationship":"Source-stated"},{"text":"At conventional significance thresholds, the authors estimate a 66 percent posterior probability of a false discovery.","excerpt":"At conventional significance thresholds, the posterior probability of a false discovery reaches 66 percent.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","relationship":"Source-stated"},{"text":"The authors report that doubling original sample sizes reduces average false discovery rates by five percentage points; tightening the significance threshold to 0.005 reduces them more, but results in fewer total discoveries.","excerpt":"Doubling original sample sizes lowers the average false discovery rate by five percentage points, whereas tightening the significance threshold to 0.005 yields a larger reduction at the cost of total discoveries.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","relationship":"Source-stated"},{"text":"Their model of replication funding under a fixed budget and evidence standard places replications in an intermediate interval of original test statistics; optimal replication sample size is single-peaked in the original t-statistic.","excerpt":"replications concentrate in an intermediate interval of original test statistics, and the optimal replication sample size is single-peaked in the original t-statistic.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","relationship":"Source-stated"},{"text":"Illustrative calibrations estimate that successful replications reduce aggregate false discovery rates to between 1.8 and 7.8 percent, depending on the evidence standard.","excerpt":"Illustrative calibrations show that successful replications reduce aggregate false discovery rates to 1.8–7.8 percent, depending on the evidence standard.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","checkedAt":"2026-11-10T22:06:08.866Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How do the paper’s replication-policy results vary across evidence standards?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Marginal Revolution introduces a paper arguing many development-economics RCTs are underpowered","summary":"Marginal Revolution introduces a paper by Qingyuan Chai, Kevin Lang and Liang Zhong on the statistical credibility of randomized controlled trials (RCTs) in development economics. The authors report that actual t-statistics peak below the theoretical benchmark expected under 80 percent power, but above values associated with p-hacking; they say this suggests many RCTs are underpowered. Using an empirical Bayes mixture model that accounts for selective publication, they estimate that approximately 30 percent of published rejections are false discoveries, with average directional power of 40 percent. At conventional significance thresholds, they estimate a 66 percent posterior probability of a false discovery. The paper compares two changes: doubling original sample sizes reduces the average false discovery rate by five percentage points, while tightening the significance threshold to 0.005 yields a larger reduction at the cost of total discoveries. It also models replication funding under a fixed budget and evidence standard. Under standard regularity conditions, the authors’ model allocates replications to an intermediate interval of original test statistics, with optimal replication sample size single-peaked in the original t-statistic.","question":"How do the paper’s replication-policy results vary across evidence standards?","claims":[{"text":"The authors report that t-statistics in their sample peak below the theoretical benchmark expected under 80 percent power, which they say suggests many RCTs are underpowered.","excerpt":"actual t-statistics peak well below the theoretical benchmark of 2.8 expected under 80 percent power, though above values associated with p-hacking. This suggests many RCTs are underpowered.","support":"source-stated"},{"text":"Using an empirical Bayes mixture model that accounts for selective publication, the authors estimate that approximately 30 percent of published rejections are false discoveries and average directional power is 40 percent.","excerpt":"we find that approximately 30 percent of published rejections are false discoveries, with average directional power at only 40 percent.","support":"source-stated"},{"text":"At conventional significance thresholds, the authors estimate a 66 percent posterior probability of a false discovery.","excerpt":"At conventional significance thresholds, the posterior probability of a false discovery reaches 66 percent.","support":"source-stated"},{"text":"The authors report that doubling original sample sizes reduces average false discovery rates by five percentage points; tightening the significance threshold to 0.005 reduces them more, but results in fewer total discoveries.","excerpt":"Doubling original sample sizes lowers the average false discovery rate by five percentage points, whereas tightening the significance threshold to 0.005 yields a larger reduction at the cost of total discoveries.","support":"source-stated"},{"text":"Their model of replication funding under a fixed budget and evidence standard places replications in an intermediate interval of original test statistics; optimal replication sample size is single-peaked in the original t-statistic.","excerpt":"replications concentrate in an intermediate interval of original test statistics, and the optimal replication sample size is single-peaked in the original t-statistic.","support":"source-stated"},{"text":"Illustrative calibrations estimate that successful replications reduce aggregate false discovery rates to between 1.8 and 7.8 percent, depending on the evidence standard.","excerpt":"Illustrative calibrations show that successful replications reduce aggregate false discovery rates to 1.8–7.8 percent, depending on the evidence standard.","support":"source-stated"}],"limitations":["The analysis described is limited to 745 principal-hypothesis tests from 190 articles in leading economics journals, in development economics.","The source presents the replication-policy results as a model under standard regularity conditions; its reported post-replication rates are illustrative calibrations.","The source does not specify the evidence standards behind the 1.8–7.8 percent calibration range."],"mediaId":"be51f1ea75a960f2faf4875e37e7a6f71193eea4b2f999907c642522c5f1afb8","correctionReason":null,"id":"5c652c1258b9669bdeb39e5a5ec5c425b7175a665c26b8baa588c8e5e238e456","publicationId":"5b8e4b7a-35fd-4280-8dc0-e76c304fabd4","version":1,"sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","sourceTitle":"Are RCTs underpowered? - Marginal REVOLUTION","publisher":"Marginal Revolution","publishedAt":"2026-11-10T20:05:26+00:00","originalPublished":{"value":"2026-11-10T20:05:26+00:00","precision":"instant","basis":"publisher_page"},"firstObservedAt":"2026-11-10T20:09:23.668Z","createdAt":"2026-11-10T21:02:41.306Z","historical":false,"entityIds":["marginal-revolution"],"topicIds":["power-and-infrastructure","work-and-agency"],"media":[{"id":"be51f1ea75a960f2faf4875e37e7a6f71193eea4b2f999907c642522c5f1afb8","kind":"article","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-10T21:01:11.224Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","checkedAt":"2026-11-10T22:06:08.866Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v5","mediaId":"be51f1ea75a960f2faf4875e37e7a6f71193eea4b2f999907c642522c5f1afb8","confidence":0.5800000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"27029b48850e1f3eae3d13324cc82f79ec93af8dec2432960f42f88cf6b4258c","decidedAt":"2026-11-10T21:01:13.610Z"},"provenance":{"textHash":"83baf6db00a1fdc9ea872bb0391dbdc5e52e6b112fec42bfc6631c4088cd1d89","observationId":"0d99035d657af4ff01ea8a35623c281493f8654f0a0250c70041dce916211e24","retrievedAt":"2026-11-10T21:01:10.873Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"2392e882-3905-479e-87bd-80e53ded5f72"},"reviewer":{"model":"gpt-6-luna","runId":"dbf62f7d-e31a-487d-8d37-b940a944fde5"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T21:02:41.306Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Tyler Cowen","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Tyler Cowen","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Tyler Cowen","status":"stated","provenance":{"basis":"article-jsonld","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","locator":"document/html[1]/head[0]/script[53]@2198.@graph[0].author"},"assertionId":"2017ed182dfff88b8cd6f2f9b92d429b3569a1b36316ba60209c8b2c8585c145:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T22:06:08.866Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"2017ed182dfff88b8cd6f2f9b92d429b3569a1b36316ba60209c8b2c8585c145","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","contentSha256":"a31bc10e9e3f6fb39d1c3b0f697649eb4d0a6ae050059b51e14fce512606b84f","textHash":"83baf6db00a1fdc9ea872bb0391dbdc5e52e6b112fec42bfc6631c4088cd1d89","observedAt":"2026-11-10T22:06:08.866Z"}}],"participationCoverage":{"total":2,"returned":1,"complete":false,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","snapshotId":"2017ed182dfff88b8cd6f2f9b92d429b3569a1b36316ba60209c8b2c8585c145","contentSha256":"a31bc10e9e3f6fb39d1c3b0f697649eb4d0a6ae050059b51e14fce512606b84f","textHash":"83baf6db00a1fdc9ea872bb0391dbdc5e52e6b112fec42bfc6631c4088cd1d89","observedAt":"2026-11-10T22:06:08.866Z","basis":"publisher-page-metadata","sourceLinks":4,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/5c652c1258b9669bdeb39e5a5ec5c425b7175a665c26b8baa588c8e5e238e456/source"}},"canonicalUrl":"https://escapemeridian.com/readings/marginal-revolution-introduces-a-paper-arguing-many-development-5c652c12","formats":{"html":"/readings/marginal-revolution-introduces-a-paper-arguing-many-development-5c652c12","markdown":"/readings/marginal-revolution-introduces-a-paper-arguing-many-development-5c652c12.md","json":"/readings/marginal-revolution-introduces-a-paper-arguing-many-development-5c652c12.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-rct","name":"Randomized controlled trial","explanation":"A study that randomly assigns people to groups so that differences in outcomes can be traced to the treatment.","referenceUrl":"https://en.wikipedia.org/wiki/Randomized_controlled_trial"},{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"marginal-revolution","name":"Marginal Revolution","kind":"organization","context":"Economics publication by Tyler Cowen and Alex Tabarrok.","referenceUrls":["https://marginalrevolution.com/"],"path":"/entities/marginal-revolution","json":"/entities/marginal-revolution.json","markdown":"/entities/marginal-revolution.md"},{"id":"tyler-cowen","name":"Tyler Cowen","kind":"person","context":"Economics, institutions and unexpected cross-domain questions; useful source discovery.","referenceUrls":["https://www.dwarkesh.com/p/tyler-cowen-4","https://www.dwarkesh.com/p/tyler-cowen-3","https://www.dwarkesh.com/p/tyler-cowen-2","https://marginalrevolution.com/marginalrevolution/2026/10/saturday-assorted-links-583.html?utm_source=rss&utm_medium=rss&utm_campaign=saturday-assorted-links-583","https://tylercowen.com/"],"path":"/entities/tyler-cowen","json":"/entities/tyler-cowen.json","markdown":"/entities/tyler-cowen.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"5c652c1258b9669bdeb39e5a5ec5c425b7175a665c26b8baa588c8e5e238e456","revision":"5b8e4b7a-35fd-4280-8dc0-e76c304fabd4","revisionKind":"publication-id","title":"Marginal Revolution introduces a paper arguing many development-economics RCTs are underpowered","url":"https://escapemeridian.com/readings/marginal-revolution-introduces-a-paper-arguing-many-development-5c652c12"},"source":{"type":"source","id":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","revision":"83baf6db00a1fdc9ea872bb0391dbdc5e52e6b112fec42bfc6631c4088cd1d89","revisionKind":"sha256-source-text","title":"Are RCTs underpowered? - Marginal REVOLUTION","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"5c652c1258b9669bdeb39e5a5ec5c425b7175a665c26b8baa588c8e5e238e456","title":"Marginal Revolution introduces a paper arguing many development-economics RCTs are underpowered","argument":"Marginal Revolution introduces a paper by Qingyuan Chai, Kevin Lang and Liang Zhong on the statistical credibility of randomized controlled trials (RCTs) in development economics. The authors report that actual t-statistics peak below the theoretical benchmark expected under 80 percent power, but above values associated with p-hacking; they say this suggests many RCTs are underpowered. Using an empirical Bayes mixture model that accounts for selective publication, they estimate that approximately 30 percent of published rejections are false discoveries, with average directional power of 40 percent. At conventional significance thresholds, they estimate a 66 percent posterior probability of a false discovery. The paper compares two changes: doubling original sample sizes reduces the average false discovery rate by five percentage points, while tightening the significance threshold to 0.005 yields a larger reduction at the cost of total discoveries. It also models replication funding under a fixed budget and evidence standard. Under standard regularity conditions, the authors’ model allocates replications to an intermediate interval of original test statistics, with optimal replication sample size single-peaked in the original t-statistic.","topicIds":["power-and-infrastructure","work-and-agency"],"domain":"work","claimType":"attributed reading","limits":"The analysis described is limited to 745 principal-hypothesis tests from 190 articles in leading economics journals, in development economics. The source presents the replication-policy results as a model under standard regularity conditions; its reported post-replication rates are illustrative calibrations. The source does not specify the evidence standards behind the 1.8–7.8 percent calibration range.","speaker":{"name":"Tyler Cowen","roleAtSource":"Article author"},"source":{"title":"Are RCTs underpowered? - Marginal REVOLUTION","publisher":"Marginal Revolution","format":"Article","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","publishedAt":"2026-11-10T20:05:26+00:00","dateBasis":"Date stated by the publisher."},"evidence":[{"text":"The authors report that t-statistics in their sample peak below the theoretical benchmark expected under 80 percent power, which they say suggests many RCTs are underpowered.","excerpt":"actual t-statistics peak well below the theoretical benchmark of 2.8 expected under 80 percent power, though above values associated with p-hacking. This suggests many RCTs are underpowered.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","relationship":"Source-stated"},{"text":"Using an empirical Bayes mixture model that accounts for selective publication, the authors estimate that approximately 30 percent of published rejections are false discoveries and average directional power is 40 percent.","excerpt":"we find that approximately 30 percent of published rejections are false discoveries, with average directional power at only 40 percent.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","relationship":"Source-stated"},{"text":"At conventional significance thresholds, the authors estimate a 66 percent posterior probability of a false discovery.","excerpt":"At conventional significance thresholds, the posterior probability of a false discovery reaches 66 percent.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","relationship":"Source-stated"},{"text":"The authors report that doubling original sample sizes reduces average false discovery rates by five percentage points; tightening the significance threshold to 0.005 reduces them more, but results in fewer total discoveries.","excerpt":"Doubling original sample sizes lowers the average false discovery rate by five percentage points, whereas tightening the significance threshold to 0.005 yields a larger reduction at the cost of total discoveries.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","relationship":"Source-stated"},{"text":"Their model of replication funding under a fixed budget and evidence standard places replications in an intermediate interval of original test statistics; optimal replication sample size is single-peaked in the original t-statistic.","excerpt":"replications concentrate in an intermediate interval of original test statistics, and the optimal replication sample size is single-peaked in the original t-statistic.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","relationship":"Source-stated"},{"text":"Illustrative calibrations estimate that successful replications reduce aggregate false discovery rates to between 1.8 and 7.8 percent, depending on the evidence standard.","excerpt":"Illustrative calibrations show that successful replications reduce aggregate false discovery rates to 1.8–7.8 percent, depending on the evidence standard.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","checkedAt":"2026-11-10T22:06:08.866Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How do the paper’s replication-policy results vary across evidence standards?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Marginal Revolution introduces a paper arguing many development-economics RCTs are underpowered","summary":"Marginal Revolution introduces a paper by Qingyuan Chai, Kevin Lang and Liang Zhong on the statistical credibility of randomized controlled trials (RCTs) in development economics. The authors report that actual t-statistics peak below the theoretical benchmark expected under 80 percent power, but above values associated with p-hacking; they say this suggests many RCTs are underpowered. Using an empirical Bayes mixture model that accounts for selective publication, they estimate that approximately 30 percent of published rejections are false discoveries, with average directional power of 40 percent. At conventional significance thresholds, they estimate a 66 percent posterior probability of a false discovery. The paper compares two changes: doubling original sample sizes reduces the average false discovery rate by five percentage points, while tightening the significance threshold to 0.005 yields a larger reduction at the cost of total discoveries. It also models replication funding under a fixed budget and evidence standard. Under standard regularity conditions, the authors’ model allocates replications to an intermediate interval of original test statistics, with optimal replication sample size single-peaked in the original t-statistic.","question":"How do the paper’s replication-policy results vary across evidence standards?","claims":[{"text":"The authors report that t-statistics in their sample peak below the theoretical benchmark expected under 80 percent power, which they say suggests many RCTs are underpowered.","excerpt":"actual t-statistics peak well below the theoretical benchmark of 2.8 expected under 80 percent power, though above values associated with p-hacking. This suggests many RCTs are underpowered.","support":"source-stated"},{"text":"Using an empirical Bayes mixture model that accounts for selective publication, the authors estimate that approximately 30 percent of published rejections are false discoveries and average directional power is 40 percent.","excerpt":"we find that approximately 30 percent of published rejections are false discoveries, with average directional power at only 40 percent.","support":"source-stated"},{"text":"At conventional significance thresholds, the authors estimate a 66 percent posterior probability of a false discovery.","excerpt":"At conventional significance thresholds, the posterior probability of a false discovery reaches 66 percent.","support":"source-stated"},{"text":"The authors report that doubling original sample sizes reduces average false discovery rates by five percentage points; tightening the significance threshold to 0.005 reduces them more, but results in fewer total discoveries.","excerpt":"Doubling original sample sizes lowers the average false discovery rate by five percentage points, whereas tightening the significance threshold to 0.005 yields a larger reduction at the cost of total discoveries.","support":"source-stated"},{"text":"Their model of replication funding under a fixed budget and evidence standard places replications in an intermediate interval of original test statistics; optimal replication sample size is single-peaked in the original t-statistic.","excerpt":"replications concentrate in an intermediate interval of original test statistics, and the optimal replication sample size is single-peaked in the original t-statistic.","support":"source-stated"},{"text":"Illustrative calibrations estimate that successful replications reduce aggregate false discovery rates to between 1.8 and 7.8 percent, depending on the evidence standard.","excerpt":"Illustrative calibrations show that successful replications reduce aggregate false discovery rates to 1.8–7.8 percent, depending on the evidence standard.","support":"source-stated"}],"limitations":["The analysis described is limited to 745 principal-hypothesis tests from 190 articles in leading economics journals, in development economics.","The source presents the replication-policy results as a model under standard regularity conditions; its reported post-replication rates are illustrative calibrations.","The source does not specify the evidence standards behind the 1.8–7.8 percent calibration range."],"mediaId":"be51f1ea75a960f2faf4875e37e7a6f71193eea4b2f999907c642522c5f1afb8","correctionReason":null,"id":"5c652c1258b9669bdeb39e5a5ec5c425b7175a665c26b8baa588c8e5e238e456","publicationId":"5b8e4b7a-35fd-4280-8dc0-e76c304fabd4","version":1,"sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","sourceTitle":"Are RCTs underpowered? - Marginal REVOLUTION","publisher":"Marginal Revolution","publishedAt":"2026-11-10T20:05:26+00:00","originalPublished":{"value":"2026-11-10T20:05:26+00:00","precision":"instant","basis":"publisher_page"},"firstObservedAt":"2026-11-10T20:09:23.668Z","createdAt":"2026-11-10T21:02:41.306Z","historical":false,"entityIds":["marginal-revolution"],"topicIds":["power-and-infrastructure","work-and-agency"],"media":[{"id":"be51f1ea75a960f2faf4875e37e7a6f71193eea4b2f999907c642522c5f1afb8","kind":"article","url":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-10T21:01:11.224Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","checkedAt":"2026-11-10T22:06:08.866Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v5","mediaId":"be51f1ea75a960f2faf4875e37e7a6f71193eea4b2f999907c642522c5f1afb8","confidence":0.5800000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"27029b48850e1f3eae3d13324cc82f79ec93af8dec2432960f42f88cf6b4258c","decidedAt":"2026-11-10T21:01:13.610Z"},"provenance":{"textHash":"83baf6db00a1fdc9ea872bb0391dbdc5e52e6b112fec42bfc6631c4088cd1d89","observationId":"0d99035d657af4ff01ea8a35623c281493f8654f0a0250c70041dce916211e24","retrievedAt":"2026-11-10T21:01:10.873Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"2392e882-3905-479e-87bd-80e53ded5f72"},"reviewer":{"model":"gpt-6-luna","runId":"dbf62f7d-e31a-487d-8d37-b940a944fde5"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T21:02:41.306Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Tyler Cowen","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Tyler Cowen","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Tyler Cowen","status":"stated","provenance":{"basis":"article-jsonld","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","locator":"document/html[1]/head[0]/script[53]@2198.@graph[0].author"},"assertionId":"2017ed182dfff88b8cd6f2f9b92d429b3569a1b36316ba60209c8b2c8585c145:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T22:06:08.866Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"2017ed182dfff88b8cd6f2f9b92d429b3569a1b36316ba60209c8b2c8585c145","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","contentSha256":"a31bc10e9e3f6fb39d1c3b0f697649eb4d0a6ae050059b51e14fce512606b84f","textHash":"83baf6db00a1fdc9ea872bb0391dbdc5e52e6b112fec42bfc6631c4088cd1d89","observedAt":"2026-11-10T22:06:08.866Z"}}],"participationCoverage":{"total":2,"returned":1,"complete":false,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/are-rcts-underpowered.html","snapshotId":"2017ed182dfff88b8cd6f2f9b92d429b3569a1b36316ba60209c8b2c8585c145","contentSha256":"a31bc10e9e3f6fb39d1c3b0f697649eb4d0a6ae050059b51e14fce512606b84f","textHash":"83baf6db00a1fdc9ea872bb0391dbdc5e52e6b112fec42bfc6631c4088cd1d89","observedAt":"2026-11-10T22:06:08.866Z","basis":"publisher-page-metadata","sourceLinks":4,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/5c652c1258b9669bdeb39e5a5ec5c425b7175a665c26b8baa588c8e5e238e456/source"}}}],"audits":[{"publicationId":"5b8e4b7a-35fd-4280-8dc0-e76c304fabd4","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The draft correctly presents this as a Marginal Revolution post quoting a new paper by Qingyuan Chai, Kevin Lang and Liang Zhong. It attributes the findings to the authors throughout and does not present them as established fact or as Escape Meridian's view.","excerpt":"That is from a new paper by Qingyuan Chai, Kevin Lang & Liang Zhong."},{"dimension":"summary","status":"uncertain","explanation":"The draft is faithful but hard for non-specialists to follow. Central terms are never explained in plain language or flagged for tooltips: underpowered, t-statistics, 80 percent power, p-hacking, empirical Bayes mixture model, directional power, posterior probability, regularity conditions and single-peaked. It also drops the 2.8 benchmark and softens 'well below' to 'below'. It does not say why the findings matter, for example that published development-economics results may be less reliable than they appear. The smallest repair is to restore the 2.8 benchmark and 'well below', and to add plain explanations or tooltips for power, false discovery and p-hacking.","excerpt":"actual t-statistics peak well below the theoretical benchmark of 2.8 expected under 80 percent power"},{"dimension":"date","status":"pass","explanation":"originalPublished (2026-11-10T20:05:26Z, publisher page) matches the feed entry for this Marginal Revolution post, which is the source being read. The post calls the paper new. No earlier first-publication date for the post appears in the evidence.","excerpt":null},{"dimension":"support","status":"pass","explanation":"Each claim matches the quoted abstract: the 30 percent false discoveries, the 40 percent directional power, the 66 percent posterior probability, the sample-size and threshold comparison, the replication allocation, and the 1.8–7.8 percent calibrations. The limitations accurately reflect the scope and the model-based nature of the replication results. The question follows from the evidence and implies no unsupported benefits or costs.","excerpt":"Using 745 principal-hypothesis tests from 190 articles in leading economics journals"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the original Marginal Revolution post, the most direct access to the source.","excerpt":null}],"createdAt":"2026-11-10T21:02:41.306Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}