· Nov 20, 2026
Marginal Revolution: AIs vs. humans in economics papers
Markdown · JSON · Original source
’s post excerpts The Economist’s account of studies comparing AI and human work in economics and related social sciences. It reports that an AI verifier found severe errors in 842 of 1,000 APE papers, including errors that invalidated headline results. In a separate review, The Economist and Refine examined 100 papers accepted by top economics journals and found at least one grave error in one in ten. The post warns that these exercises are not directly comparable, so their figures are not a head-to-head measure of AI and human performance. The comparison is about more than error counts. One study found AI lowers the cost of checking a candidate flaw, but “does not yet reliably substitute for the human judgment that locates the flaw in the first place”. Another found that the difference between APE papers and published papers was mostly in ideas, not execution. In a study grading AI and human papers, AI did better in crowded fields with lots of existing research to digest, while human performance did not vary with literature size. A reproduction study found discrepancies between original social-science papers and AI versions were due in roughly equal measure to problems with AI and human work. But the post says that as the AI model improved, the balance tilted against humans.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
· Article
Source details
Original source date: Nov 20, 2026 (2026-11-20T05:53:10+00:00).
Date basis: Date stated by the publisher.
Added to Escape Meridian: 2026-11-20T13:02:12.337Z. First observed: 2026-11-20T05:55:02.361Z. Retrieved: 2026-11-20T13:00:55.147Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The post summarizes multiple studies but does not provide their full methods or enough detail to assess how the error categories were defined. The post explicitly says the two error-checking exercises are not directly comparable; their reported error rates should not be treated as a head-to-head result. The reproduction study excerpt does not specify how much better AI models changed the balance or whether the trend continues beyond the models studied.
Open question
In the reproduction study, does the balance continue to tilt against human work as AI models improve beyond the models tested?
Source-stated
The post reports that an AI verifier found at least one severe error in 842 of 1,000 APE papers.
None was error-free; 842 contained at least one severe error, including ones invalidating headline results.
Source-stated
A review of a random sample of 100 papers accepted by leading economics journals flagged at least one grave error in one in ten.
One in ten papers contained at least one grave error
Source-stated
The post cautions that the two error-checking exercises are not directly comparable.
The two exercises are not directly comparable
Source-stated
A study cited in the post found that AI can help check a candidate flaw, but does not reliably replace human judgment in finding it.
“does not yet reliably substitute for the human judgment that locates the flaw in the first place”
Source-stated
A cited study found AI did better in crowded fields, while human performance was the same regardless of literature size.
AI did better in crowded fields with lots of existing research to digest. Humans performed at the same level regardless of the size of the literature.
Source-stated
In a reproduction study, discrepancies were attributed roughly equally to problems with AI and human work, with better models shifting the balance against humans.
The discrepancies between the original papers and the AI versions were due in roughly equal measure to problems with the AI and human work. But the better the AI model, the more the balance tilted against humans.
Source and contributor context
Tyler Cowen · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Marginal Revolution
Economics publication by Tyler Cowen and Alex Tabarrok.
Tyler Cowen
Economics, institutions and unexpected cross-domain questions; useful source discovery.
Editorial provenance
Version 2. Publication ID: e92a98fe-9438-4789-80f8-3d17db77f4fa.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-20T13:02:12.337Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: useful_perspective.
Observation: 27ca1ae721f20495b18f258f1928da946750e2f91bd021a27af8c49e3d41539f. Source text hash: 0b5b4ee61abbf99f2993083aefc5a0e2c846f158142d533fd808fd2843c55cbe.
Correction reason: Revised the summary and open question to clarify the studies’ distinct findings and frame the reproduction study’s future trend as unresolved.
Revision and source-check history
Version 2 · 2026-11-20T13:02:12.337Z · Revised the summary and open question to clarify the studies’ distinct findings and frame the reproduction study’s future trend as unresolved.
Marginal Revolution: AIs vs. humans in economics papers
’s post excerpts The Economist’s account of studies comparing AI and human work in economics and related social sciences. It reports that an AI verifier found severe errors in 842 of 1,000 APE papers, including errors that invalidated headline results. In a separate review, The Economist and Refine examined 100 papers accepted by top economics journals and found at least one grave error in one in ten. The post warns that these exercises are not directly comparable, so their figures are not a head-to-head measure of AI and human performance. The comparison is about more than error counts. One study found AI lowers the cost of checking a candidate flaw, but “does not yet reliably substitute for the human judgment that locates the flaw in the first place”. Another found that the difference between APE papers and published papers was mostly in ideas, not execution. In a study grading AI and human papers, AI did better in crowded fields with lots of existing research to digest, while human performance did not vary with literature size. A reproduction study found discrepancies between original social-science papers and AI versions were due in roughly equal measure to problems with AI and human work. But the post says that as the AI model improved, the balance tilted against humans.
Original source
· Article
Source details
Original source date: Nov 20, 2026 (2026-11-20T05:53:10+00:00).
Date basis: Date stated by the publisher.
Added to Escape Meridian: 2026-11-20T13:02:12.337Z. First observed: 2026-11-20T05:55:02.361Z. Retrieved: 2026-11-20T13:00:55.147Z.
Limits
The post summarizes multiple studies but does not provide their full methods or enough detail to assess how the error categories were defined. The post explicitly says the two error-checking exercises are not directly comparable; their reported error rates should not be treated as a head-to-head result. The reproduction study excerpt does not specify how much better AI models changed the balance or whether the trend continues beyond the models studied.
Open question
In the reproduction study, does the balance continue to tilt against human work as AI models improve beyond the models tested?
Source-stated
The post reports that an AI verifier found at least one severe error in 842 of 1,000 APE papers.
None was error-free; 842 contained at least one severe error, including ones invalidating headline results.
Source-stated
A review of a random sample of 100 papers accepted by leading economics journals flagged at least one grave error in one in ten.
One in ten papers contained at least one grave error
Source-stated
The post cautions that the two error-checking exercises are not directly comparable.
The two exercises are not directly comparable
Source-stated
A study cited in the post found that AI can help check a candidate flaw, but does not reliably replace human judgment in finding it.
“does not yet reliably substitute for the human judgment that locates the flaw in the first place”
Source-stated
A cited study found AI did better in crowded fields, while human performance was the same regardless of literature size.
AI did better in crowded fields with lots of existing research to digest. Humans performed at the same level regardless of the size of the literature.
Source-stated
In a reproduction study, discrepancies were attributed roughly equally to problems with AI and human work, with better models shifting the balance against humans.
The discrepancies between the original papers and the AI versions were due in roughly equal measure to problems with the AI and human work. But the better the AI model, the more the balance tilted against humans.
Source and contributor context
Tyler Cowen · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Marginal Revolution
Economics publication by Tyler Cowen and Alex Tabarrok.
Tyler Cowen
Economics, institutions and unexpected cross-domain questions; useful source discovery.
Editorial provenance
Version 2. Publication ID: e92a98fe-9438-4789-80f8-3d17db77f4fa.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-20T13:02:12.337Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: useful_perspective.
Observation: 27ca1ae721f20495b18f258f1928da946750e2f91bd021a27af8c49e3d41539f. Source text hash: 0b5b4ee61abbf99f2993083aefc5a0e2c846f158142d533fd808fd2843c55cbe.
Correction reason: Revised the summary and open question to clarify the studies’ distinct findings and frame the reproduction study’s future trend as unresolved.
Version 1 · 2026-11-20T06:01:45.012Z
Marginal Revolution: AIs vs. humans in economics papers
’s post compares studies of AI and human work in economics and related social sciences. It reports that an AI verifier found severe errors in 842 of 1,000 APE papers, while a review of 100 papers accepted by leading economics journals flagged at least one grave error in one in ten. The post cautions that these exercises are not directly comparable. Its account is not a simple contest over error rates: one study found AI can lower the cost of checking a candidate flaw, but “does not yet reliably substitute for the human judgment that locates the flaw in the first place”. Another comparison found that the difference between APE and published papers was mostly in ideas, not execution. AI did better in crowded research fields, while human performance was the same regardless of literature size. In a separate reproduction exercise, discrepancies between AI and human versions of social-science papers were due in roughly equal measure to problems with each. But the better the AI model, the more the balance tilted against humans. The studies point to differences across research tasks, while leaving open how that balance may change as models improve.
Original source
· Article
Source details
Original source date: Nov 20, 2026 (2026-11-20T05:53:10+00:00).
Date basis: Date stated by the publisher.
Added to Escape Meridian: 2026-11-20T06:01:45.012Z. First observed: 2026-11-20T05:55:02.361Z. Retrieved: 2026-11-20T06:00:51.272Z.
Limits
The post summarizes multiple studies but does not provide their full methods or enough detail to assess how the error categories were defined. The post explicitly says the two error-checking exercises are not directly comparable; their reported error rates should not be treated as a head-to-head result. The reproduction study excerpt does not specify how much better AI models changed the balance or whether the trend continues beyond the models studied.
Open question
As AI models improve, does the balance of discrepancies in reproducing social-science papers continue to shift against human work?
Source-stated
The post reports that an AI verifier found at least one severe error in 842 of 1,000 APE papers.
None was error-free; 842 contained at least one severe error, including ones invalidating headline results.
Source-stated
A review of a random sample of 100 papers accepted by leading economics journals flagged at least one grave error in one in ten.
One in ten papers contained at least one grave error
Source-stated
The post cautions that the two error-checking exercises are not directly comparable.
The two exercises are not directly comparable
Source-stated
A study cited in the post found that AI can help check a candidate flaw, but does not reliably replace human judgment in finding it.
“does not yet reliably substitute for the human judgment that locates the flaw in the first place”
Source-stated
A cited study found AI did better in crowded fields, while human performance was the same regardless of literature size.
AI did better in crowded fields with lots of existing research to digest. Humans performed at the same level regardless of the size of the literature.
Source-stated
In a reproduction study, discrepancies were attributed roughly equally to problems with AI and human work, with better models shifting the balance against humans.
The discrepancies between the original papers and the AI versions were due in roughly equal measure to problems with the AI and human work. But the better the AI model, the more the balance tilted against humans.
Source and contributor context
Tyler Cowen · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Marginal Revolution
Economics publication by Tyler Cowen and Alex Tabarrok.
Tyler Cowen
Economics, institutions and unexpected cross-domain questions; useful source discovery.
Editorial provenance
Version 1. Publication ID: 5bf85a5b-d449-421d-8a81-ad11ff24720c.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-20T06:01:45.012Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: useful_perspective.
Observation: ce415bffa0e69325137c0324b24f8847d0729d68c4d32a4bba28c687b526ec07. Source text hash: 01877528884b64ff985d1d1213bce53a6896a2f3de9f4753e846a2452f601cfd.
uncertain · 2026-11-20T06:01:45.012Z · anthropic/claude-opus-5.5
Publication: 5bf85a5b-d449-421d-8a81-ad11ff24720c
attribution — uncertain: The reading presents everything as Marginal Revolution's own post comparing studies, but the text is an excerpt from The Economist. The 100-paper review was run by The Economist with Refine, and the 'not directly comparable' caution is The Economist's wording, not MR's. The named researchers (Yanagizawa-Drott and Willner, Toda, Zampa, Kohler) are also omitted. Smallest repair: say MR is quoting The Economist, and credit the 100-paper review to The Economist and Refine.
Here is more from The Economist, interesting throughout.
summary — uncertain: 'APE papers' is the central term and is never explained. A general reader cannot tell that these are AI-generated (agentic research) papers, so the headline 842/1,000 comparison is hard to follow. The source's own framing is also flattened: it says humans' lower error rate 'might give them some hope' and that people retain the edge in ideas. Repair: briefly gloss APE as AI-agent-produced economics papers and keep that hedged conclusion.
Is agentic research any good?
date — pass: The originalPublished value, 2026-11-20T05:53:10Z, comes from the publisher page and matches the feed timestamp. No earlier publication is indicated, and the draft makes no conflicting date claim.
support — uncertain: The claims are grounded in the text, but several scope conditions are dropped. First, the crowded-fields result comes from an AI referee grading the papers, and the reading presents it without that condition. Second, Toda's finding applied to economic theory papers only. Third, the claim drops 'yet' from 'does not yet reliably substitute'. Repairs: note that an AI referee did the grading, restrict Toda's finding to theory papers, and keep 'yet'. The question is fairly derived from the reproduction finding.
Michele Zampa of the Geneva Graduate Institute used an AI referee to grade 981 AI papers and 41 human ones
media — pass: The selected media is the verified direct link to the original Marginal Revolution post. It is the most relevant available item.
uncertain · 2026-11-20T13:02:12.337Z · anthropic/claude-opus-5.5
Publication: e92a98fe-9438-4789-80f8-3d17db77f4fa
attribution — pass: The draft correctly presents this as a Marginal Revolution post excerpting The Economist. It attributes the Refine review to The Economist and Refine, and each study finding to the cited researchers or studies, without presenting them as established fact.
Here is more from The Economist, interesting throughout.
summary — uncertain: The central term APE is never explained, so a non-specialist cannot tell that the 842-of-1,000 figure concerns AI-generated agentic research papers. The comparison depends on this. Smallest repair: on first use, describe APE papers as AI-produced (agentic) research papers. The draft also drops the source's stated takeaway that humans' lower error rate offers some hope and that people retain the edge in ideas. Restoring that clause would preserve the source's framing.
The two exercises are not directly comparable, but human economists’ lower error rate might give them some hope. In ideas, too, people retain the edge.
date — pass: The originalPublished value of 2026-11-20T05:53:10Z comes from the publisher page and matches the feed entry for this Marginal Revolution post. The draft makes no conflicting date claim.
support — pass: All claims match the excerpted text, including the figures, quotations and the non-comparability caveat. The phrase 'as the AI model improved' slightly reframes 'the better the AI model', but it stays within the source's meaning. The question is properly framed as unresolved, and the limitations correctly note the missing detail.
But the better the AI model, the more the balance tilted against humans.
media — pass: The selected media ID is the verified direct link to the original Marginal Revolution post, which is the source itself.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
Economics, institutions and unexpected cross-domain questions; useful source discovery.
Explore Tyler CowenEconomics publication by Tyler Cowen and Alex Tabarrok.
Explore Marginal RevolutionEconomics publication by Tyler Cowen and Alex Tabarrok.
Explore Marginal RevolutionEconomics publication by Tyler Cowen and Alex Tabarrok.
Explore Marginal RevolutionEconomics publication by Tyler Cowen and Alex Tabarrok.
Explore Marginal RevolutionEconomics publication by Tyler Cowen and Alex Tabarrok.
Explore Marginal RevolutionEconomics publication by Tyler Cowen and Alex Tabarrok.
Explore Marginal Revolution