{"id":"4ccdc7a62f2032c232b637124169a98de568a28f1fa00923a789295a76b04d22","title":"Marginal Revolution: AIs vs. humans in economics papers","argument":"Marginal Revolution’s post excerpts The Economist’s account of studies comparing AI and human work in economics and related social sciences. It reports that an AI verifier found severe errors in 842 of 1,000 APE papers, including errors that invalidated headline results. In a separate review, The Economist and Refine examined 100 papers accepted by top economics journals and found at least one grave error in one in ten. The post warns that these exercises are not directly comparable, so their figures are not a head-to-head measure of AI and human performance.\n\nThe comparison is about more than error counts. One study found AI lowers the cost of checking a candidate flaw, but “does not yet reliably substitute for the human judgment that locates the flaw in the first place”. Another found that the difference between APE papers and published papers was mostly in ideas, not execution. In a study grading AI and human papers, AI did better in crowded fields with lots of existing research to digest, while human performance did not vary with literature size.\n\nA reproduction study found discrepancies between original social-science papers and AI versions were due in roughly equal measure to problems with AI and human work. But the post says that as the AI model improved, the balance tilted against humans.","topicIds":["power-and-infrastructure","work-and-agency"],"domain":"work","claimType":"attributed reading","limits":"The post summarizes multiple studies but does not provide their full methods or enough detail to assess how the error categories were defined. The post explicitly says the two error-checking exercises are not directly comparable; their reported error rates should not be treated as a head-to-head result. The reproduction study excerpt does not specify how much better AI models changed the balance or whether the trend continues beyond the models studied.","speaker":{"name":"Tyler Cowen","roleAtSource":"Article author"},"source":{"title":"AIs vs. humans in economics papers - Marginal REVOLUTION","publisher":"Marginal Revolution","format":"Article","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","publishedAt":"2026-11-20T05:53:10+00:00","dateBasis":"Date stated by the publisher."},"evidence":[{"text":"The post reports that an AI verifier found at least one severe error in 842 of 1,000 APE papers.","excerpt":"None was error-free; 842 contained at least one severe error, including ones invalidating headline results.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"A review of a random sample of 100 papers accepted by leading economics journals flagged at least one grave error in one in ten.","excerpt":"One in ten papers contained at least one grave error","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"The post cautions that the two error-checking exercises are not directly comparable.","excerpt":"The two exercises are not directly comparable","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"A study cited in the post found that AI can help check a candidate flaw, but does not reliably replace human judgment in finding it.","excerpt":"“does not yet reliably substitute for the human judgment that locates the flaw in the first place”","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"A cited study found AI did better in crowded fields, while human performance was the same regardless of literature size.","excerpt":"AI did better in crowded fields with lots of existing research to digest. Humans performed at the same level regardless of the size of the literature.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"In a reproduction study, discrepancies were attributed roughly equally to problems with AI and human work, with better models shifting the balance against humans.","excerpt":"The discrepancies between the original papers and the AI versions were due in roughly equal measure to problems with the AI and human work. But the better the AI model, the more the balance tilted against humans.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","checkedAt":"2026-11-20T21:34:11.456Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"In the reproduction study, does the balance continue to tilt against human work as AI models improve beyond the models tested?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Marginal Revolution: AIs vs. humans in economics papers","summary":"Marginal Revolution’s post excerpts The Economist’s account of studies comparing AI and human work in economics and related social sciences. It reports that an AI verifier found severe errors in 842 of 1,000 APE papers, including errors that invalidated headline results. In a separate review, The Economist and Refine examined 100 papers accepted by top economics journals and found at least one grave error in one in ten. The post warns that these exercises are not directly comparable, so their figures are not a head-to-head measure of AI and human performance.\n\nThe comparison is about more than error counts. One study found AI lowers the cost of checking a candidate flaw, but “does not yet reliably substitute for the human judgment that locates the flaw in the first place”. Another found that the difference between APE papers and published papers was mostly in ideas, not execution. In a study grading AI and human papers, AI did better in crowded fields with lots of existing research to digest, while human performance did not vary with literature size.\n\nA reproduction study found discrepancies between original social-science papers and AI versions were due in roughly equal measure to problems with AI and human work. But the post says that as the AI model improved, the balance tilted against humans.","question":"In the reproduction study, does the balance continue to tilt against human work as AI models improve beyond the models tested?","claims":[{"text":"The post reports that an AI verifier found at least one severe error in 842 of 1,000 APE papers.","excerpt":"None was error-free; 842 contained at least one severe error, including ones invalidating headline results.","support":"source-stated"},{"text":"A review of a random sample of 100 papers accepted by leading economics journals flagged at least one grave error in one in ten.","excerpt":"One in ten papers contained at least one grave error","support":"source-stated"},{"text":"The post cautions that the two error-checking exercises are not directly comparable.","excerpt":"The two exercises are not directly comparable","support":"source-stated"},{"text":"A study cited in the post found that AI can help check a candidate flaw, but does not reliably replace human judgment in finding it.","excerpt":"“does not yet reliably substitute for the human judgment that locates the flaw in the first place”","support":"source-stated"},{"text":"A cited study found AI did better in crowded fields, while human performance was the same regardless of literature size.","excerpt":"AI did better in crowded fields with lots of existing research to digest. Humans performed at the same level regardless of the size of the literature.","support":"source-stated"},{"text":"In a reproduction study, discrepancies were attributed roughly equally to problems with AI and human work, with better models shifting the balance against humans.","excerpt":"The discrepancies between the original papers and the AI versions were due in roughly equal measure to problems with the AI and human work. But the better the AI model, the more the balance tilted against humans.","support":"source-stated"}],"limitations":["The post summarizes multiple studies but does not provide their full methods or enough detail to assess how the error categories were defined.","The post explicitly says the two error-checking exercises are not directly comparable; their reported error rates should not be treated as a head-to-head result.","The reproduction study excerpt does not specify how much better AI models changed the balance or whether the trend continues beyond the models studied."],"mediaId":"7a934f0174d68735bf149ec3cd47a05260a42c2c1e1b1d4a1dffd31c956247aa","correctionReason":"Revised the summary and open question to clarify the studies’ distinct findings and frame the reproduction study’s future trend as unresolved.","id":"4ccdc7a62f2032c232b637124169a98de568a28f1fa00923a789295a76b04d22","publicationId":"e92a98fe-9438-4789-80f8-3d17db77f4fa","version":2,"sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","sourceTitle":"AIs vs. humans in economics papers - Marginal REVOLUTION","publisher":"Marginal Revolution","publishedAt":"2026-11-20T05:53:10+00:00","originalPublished":{"value":"2026-11-20T05:53:10+00:00","precision":"instant","basis":"publisher_page"},"firstObservedAt":"2026-11-20T05:55:02.361Z","createdAt":"2026-11-20T13:02:12.337Z","historical":false,"entityIds":["marginal-revolution"],"topicIds":["power-and-infrastructure","work-and-agency"],"media":[{"id":"7a934f0174d68735bf149ec3cd47a05260a42c2c1e1b1d4a1dffd31c956247aa","kind":"article","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-20T13:00:55.425Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","checkedAt":"2026-11-20T21:34:11.456Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v9","mediaId":"7a934f0174d68735bf149ec3cd47a05260a42c2c1e1b1d4a1dffd31c956247aa","confidence":0.98,"provider":"typesafe","model":"jev-1.13.0","inputHash":"e270629981d774b8242dabcddf8f41414390f0b5949fef423f780ae0c85c9b44","decidedAt":"2026-11-20T13:00:57.043Z"},"provenance":{"textHash":"0b5b4ee61abbf99f2993083aefc5a0e2c846f158142d533fd808fd2843c55cbe","observationId":"27ca1ae721f20495b18f258f1928da946750e2f91bd021a27af8c49e3d41539f","retrievedAt":"2026-11-20T13:00:55.147Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"a20ae0d8-f71a-4ed6-8a92-139307fcee9a"},"reviewer":{"model":"gpt-6-luna","runId":"2754933b-fc67-4e94-8245-0e4092f19fba"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-20T13:02:12.337Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Tyler Cowen","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Tyler Cowen","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Tyler Cowen","status":"stated","provenance":{"basis":"article-jsonld","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","locator":"document/html[1]/head[0]/script[55]@2300.@graph[0].author"},"assertionId":"0bb03750074bae7d0fd4db940f8e73070889f2cd8a2f777ba23d0ea62dbef235:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-20T21:34:11.456Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0bb03750074bae7d0fd4db940f8e73070889f2cd8a2f777ba23d0ea62dbef235","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","contentSha256":"ebe941779005072656e837dcacd57c8f5f92686fc724a998163813293320f54f","textHash":"0b5b4ee61abbf99f2993083aefc5a0e2c846f158142d533fd808fd2843c55cbe","observedAt":"2026-11-20T21:34:11.456Z"}}],"participationCoverage":{"total":2,"returned":1,"complete":false,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","snapshotId":"0bb03750074bae7d0fd4db940f8e73070889f2cd8a2f777ba23d0ea62dbef235","contentSha256":"ebe941779005072656e837dcacd57c8f5f92686fc724a998163813293320f54f","textHash":"0b5b4ee61abbf99f2993083aefc5a0e2c846f158142d533fd808fd2843c55cbe","observedAt":"2026-11-20T21:34:11.456Z","basis":"publisher-page-metadata","sourceLinks":1,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/4ccdc7a62f2032c232b637124169a98de568a28f1fa00923a789295a76b04d22/source"}},"canonicalUrl":"https://escapemeridian.com/readings/marginal-revolution-ais-vs-humans-in-economics-papers-4ccdc7a6","formats":{"html":"/readings/marginal-revolution-ais-vs-humans-in-economics-papers-4ccdc7a6","markdown":"/readings/marginal-revolution-ais-vs-humans-in-economics-papers-4ccdc7a6.md","json":"/readings/marginal-revolution-ais-vs-humans-in-economics-papers-4ccdc7a6.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[],"entities":[{"id":"marginal-revolution","name":"Marginal Revolution","kind":"organization","context":"Economics publication by Tyler Cowen and Alex Tabarrok.","referenceUrls":["https://marginalrevolution.com/"],"path":"/entities/marginal-revolution","json":"/entities/marginal-revolution.json","markdown":"/entities/marginal-revolution.md"},{"id":"tyler-cowen","name":"Tyler Cowen","kind":"person","context":"Economics, institutions and unexpected cross-domain questions; useful source discovery.","referenceUrls":["https://www.dwarkesh.com/p/tyler-cowen-4","https://www.dwarkesh.com/p/tyler-cowen-3","https://www.dwarkesh.com/p/tyler-cowen-2","https://marginalrevolution.com/marginalrevolution/2026/10/saturday-assorted-links-583.html?utm_source=rss&utm_medium=rss&utm_campaign=saturday-assorted-links-583","https://tylercowen.com/"],"path":"/entities/tyler-cowen","json":"/entities/tyler-cowen.json","markdown":"/entities/tyler-cowen.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"4ccdc7a62f2032c232b637124169a98de568a28f1fa00923a789295a76b04d22","revision":"e92a98fe-9438-4789-80f8-3d17db77f4fa","revisionKind":"publication-id","title":"Marginal Revolution: AIs vs. humans in economics papers","url":"https://escapemeridian.com/readings/marginal-revolution-ais-vs-humans-in-economics-papers-4ccdc7a6"},"source":{"type":"source","id":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","revision":"0b5b4ee61abbf99f2993083aefc5a0e2c846f158142d533fd808fd2843c55cbe","revisionKind":"sha256-source-text","title":"AIs vs. humans in economics papers - Marginal REVOLUTION","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter."},"revisions":[{"id":"4ccdc7a62f2032c232b637124169a98de568a28f1fa00923a789295a76b04d22","title":"Marginal Revolution: AIs vs. humans in economics papers","argument":"Marginal Revolution’s post excerpts The Economist’s account of studies comparing AI and human work in economics and related social sciences. It reports that an AI verifier found severe errors in 842 of 1,000 APE papers, including errors that invalidated headline results. In a separate review, The Economist and Refine examined 100 papers accepted by top economics journals and found at least one grave error in one in ten. The post warns that these exercises are not directly comparable, so their figures are not a head-to-head measure of AI and human performance.\n\nThe comparison is about more than error counts. One study found AI lowers the cost of checking a candidate flaw, but “does not yet reliably substitute for the human judgment that locates the flaw in the first place”. Another found that the difference between APE papers and published papers was mostly in ideas, not execution. In a study grading AI and human papers, AI did better in crowded fields with lots of existing research to digest, while human performance did not vary with literature size.\n\nA reproduction study found discrepancies between original social-science papers and AI versions were due in roughly equal measure to problems with AI and human work. But the post says that as the AI model improved, the balance tilted against humans.","topicIds":["power-and-infrastructure","work-and-agency"],"domain":"work","claimType":"attributed reading","limits":"The post summarizes multiple studies but does not provide their full methods or enough detail to assess how the error categories were defined. The post explicitly says the two error-checking exercises are not directly comparable; their reported error rates should not be treated as a head-to-head result. The reproduction study excerpt does not specify how much better AI models changed the balance or whether the trend continues beyond the models studied.","speaker":{"name":"Tyler Cowen","roleAtSource":"Article author"},"source":{"title":"AIs vs. humans in economics papers - Marginal REVOLUTION","publisher":"Marginal Revolution","format":"Article","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","publishedAt":"2026-11-20T05:53:10+00:00","dateBasis":"Date stated by the publisher."},"evidence":[{"text":"The post reports that an AI verifier found at least one severe error in 842 of 1,000 APE papers.","excerpt":"None was error-free; 842 contained at least one severe error, including ones invalidating headline results.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"A review of a random sample of 100 papers accepted by leading economics journals flagged at least one grave error in one in ten.","excerpt":"One in ten papers contained at least one grave error","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"The post cautions that the two error-checking exercises are not directly comparable.","excerpt":"The two exercises are not directly comparable","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"A study cited in the post found that AI can help check a candidate flaw, but does not reliably replace human judgment in finding it.","excerpt":"“does not yet reliably substitute for the human judgment that locates the flaw in the first place”","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"A cited study found AI did better in crowded fields, while human performance was the same regardless of literature size.","excerpt":"AI did better in crowded fields with lots of existing research to digest. Humans performed at the same level regardless of the size of the literature.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"In a reproduction study, discrepancies were attributed roughly equally to problems with AI and human work, with better models shifting the balance against humans.","excerpt":"The discrepancies between the original papers and the AI versions were due in roughly equal measure to problems with the AI and human work. But the better the AI model, the more the balance tilted against humans.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","checkedAt":"2026-11-20T21:34:11.456Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"In the reproduction study, does the balance continue to tilt against human work as AI models improve beyond the models tested?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Marginal Revolution: AIs vs. humans in economics papers","summary":"Marginal Revolution’s post excerpts The Economist’s account of studies comparing AI and human work in economics and related social sciences. It reports that an AI verifier found severe errors in 842 of 1,000 APE papers, including errors that invalidated headline results. In a separate review, The Economist and Refine examined 100 papers accepted by top economics journals and found at least one grave error in one in ten. The post warns that these exercises are not directly comparable, so their figures are not a head-to-head measure of AI and human performance.\n\nThe comparison is about more than error counts. One study found AI lowers the cost of checking a candidate flaw, but “does not yet reliably substitute for the human judgment that locates the flaw in the first place”. Another found that the difference between APE papers and published papers was mostly in ideas, not execution. In a study grading AI and human papers, AI did better in crowded fields with lots of existing research to digest, while human performance did not vary with literature size.\n\nA reproduction study found discrepancies between original social-science papers and AI versions were due in roughly equal measure to problems with AI and human work. But the post says that as the AI model improved, the balance tilted against humans.","question":"In the reproduction study, does the balance continue to tilt against human work as AI models improve beyond the models tested?","claims":[{"text":"The post reports that an AI verifier found at least one severe error in 842 of 1,000 APE papers.","excerpt":"None was error-free; 842 contained at least one severe error, including ones invalidating headline results.","support":"source-stated"},{"text":"A review of a random sample of 100 papers accepted by leading economics journals flagged at least one grave error in one in ten.","excerpt":"One in ten papers contained at least one grave error","support":"source-stated"},{"text":"The post cautions that the two error-checking exercises are not directly comparable.","excerpt":"The two exercises are not directly comparable","support":"source-stated"},{"text":"A study cited in the post found that AI can help check a candidate flaw, but does not reliably replace human judgment in finding it.","excerpt":"“does not yet reliably substitute for the human judgment that locates the flaw in the first place”","support":"source-stated"},{"text":"A cited study found AI did better in crowded fields, while human performance was the same regardless of literature size.","excerpt":"AI did better in crowded fields with lots of existing research to digest. Humans performed at the same level regardless of the size of the literature.","support":"source-stated"},{"text":"In a reproduction study, discrepancies were attributed roughly equally to problems with AI and human work, with better models shifting the balance against humans.","excerpt":"The discrepancies between the original papers and the AI versions were due in roughly equal measure to problems with the AI and human work. But the better the AI model, the more the balance tilted against humans.","support":"source-stated"}],"limitations":["The post summarizes multiple studies but does not provide their full methods or enough detail to assess how the error categories were defined.","The post explicitly says the two error-checking exercises are not directly comparable; their reported error rates should not be treated as a head-to-head result.","The reproduction study excerpt does not specify how much better AI models changed the balance or whether the trend continues beyond the models studied."],"mediaId":"7a934f0174d68735bf149ec3cd47a05260a42c2c1e1b1d4a1dffd31c956247aa","correctionReason":"Revised the summary and open question to clarify the studies’ distinct findings and frame the reproduction study’s future trend as unresolved.","id":"4ccdc7a62f2032c232b637124169a98de568a28f1fa00923a789295a76b04d22","publicationId":"e92a98fe-9438-4789-80f8-3d17db77f4fa","version":2,"sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","sourceTitle":"AIs vs. humans in economics papers - Marginal REVOLUTION","publisher":"Marginal Revolution","publishedAt":"2026-11-20T05:53:10+00:00","originalPublished":{"value":"2026-11-20T05:53:10+00:00","precision":"instant","basis":"publisher_page"},"firstObservedAt":"2026-11-20T05:55:02.361Z","createdAt":"2026-11-20T13:02:12.337Z","historical":false,"entityIds":["marginal-revolution"],"topicIds":["power-and-infrastructure","work-and-agency"],"media":[{"id":"7a934f0174d68735bf149ec3cd47a05260a42c2c1e1b1d4a1dffd31c956247aa","kind":"article","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-20T13:00:55.425Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","checkedAt":"2026-11-20T21:34:11.456Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v9","mediaId":"7a934f0174d68735bf149ec3cd47a05260a42c2c1e1b1d4a1dffd31c956247aa","confidence":0.98,"provider":"typesafe","model":"jev-1.13.0","inputHash":"e270629981d774b8242dabcddf8f41414390f0b5949fef423f780ae0c85c9b44","decidedAt":"2026-11-20T13:00:57.043Z"},"provenance":{"textHash":"0b5b4ee61abbf99f2993083aefc5a0e2c846f158142d533fd808fd2843c55cbe","observationId":"27ca1ae721f20495b18f258f1928da946750e2f91bd021a27af8c49e3d41539f","retrievedAt":"2026-11-20T13:00:55.147Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"a20ae0d8-f71a-4ed6-8a92-139307fcee9a"},"reviewer":{"model":"gpt-6-luna","runId":"2754933b-fc67-4e94-8245-0e4092f19fba"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-20T13:02:12.337Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Tyler Cowen","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Tyler Cowen","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Tyler Cowen","status":"stated","provenance":{"basis":"article-jsonld","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","locator":"document/html[1]/head[0]/script[55]@2300.@graph[0].author"},"assertionId":"0bb03750074bae7d0fd4db940f8e73070889f2cd8a2f777ba23d0ea62dbef235:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-20T21:34:11.456Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0bb03750074bae7d0fd4db940f8e73070889f2cd8a2f777ba23d0ea62dbef235","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","contentSha256":"ebe941779005072656e837dcacd57c8f5f92686fc724a998163813293320f54f","textHash":"0b5b4ee61abbf99f2993083aefc5a0e2c846f158142d533fd808fd2843c55cbe","observedAt":"2026-11-20T21:34:11.456Z"}}],"participationCoverage":{"total":2,"returned":1,"complete":false,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","snapshotId":"0bb03750074bae7d0fd4db940f8e73070889f2cd8a2f777ba23d0ea62dbef235","contentSha256":"ebe941779005072656e837dcacd57c8f5f92686fc724a998163813293320f54f","textHash":"0b5b4ee61abbf99f2993083aefc5a0e2c846f158142d533fd808fd2843c55cbe","observedAt":"2026-11-20T21:34:11.456Z","basis":"publisher-page-metadata","sourceLinks":1,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/4ccdc7a62f2032c232b637124169a98de568a28f1fa00923a789295a76b04d22/source"}}},{"id":"4ccdc7a62f2032c232b637124169a98de568a28f1fa00923a789295a76b04d22","title":"Marginal Revolution: AIs vs. humans in economics papers","argument":"Marginal Revolution’s post compares studies of AI and human work in economics and related social sciences. It reports that an AI verifier found severe errors in 842 of 1,000 APE papers, while a review of 100 papers accepted by leading economics journals flagged at least one grave error in one in ten. The post cautions that these exercises are not directly comparable. Its account is not a simple contest over error rates: one study found AI can lower the cost of checking a candidate flaw, but “does not yet reliably substitute for the human judgment that locates the flaw in the first place”. Another comparison found that the difference between APE and published papers was mostly in ideas, not execution. AI did better in crowded research fields, while human performance was the same regardless of literature size. In a separate reproduction exercise, discrepancies between AI and human versions of social-science papers were due in roughly equal measure to problems with each. But the better the AI model, the more the balance tilted against humans. The studies point to differences across research tasks, while leaving open how that balance may change as models improve.","topicIds":["power-and-infrastructure","work-and-agency"],"domain":"work","claimType":"attributed reading","limits":"The post summarizes multiple studies but does not provide their full methods or enough detail to assess how the error categories were defined. The post explicitly says the two error-checking exercises are not directly comparable; their reported error rates should not be treated as a head-to-head result. The reproduction study excerpt does not specify how much better AI models changed the balance or whether the trend continues beyond the models studied.","speaker":{"name":"Tyler Cowen","roleAtSource":"Article author"},"source":{"title":"AIs vs. humans in economics papers - Marginal REVOLUTION","publisher":"Marginal Revolution","format":"Article","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","publishedAt":"2026-11-20T05:53:10+00:00","dateBasis":"Date stated by the publisher."},"evidence":[{"text":"The post reports that an AI verifier found at least one severe error in 842 of 1,000 APE papers.","excerpt":"None was error-free; 842 contained at least one severe error, including ones invalidating headline results.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"A review of a random sample of 100 papers accepted by leading economics journals flagged at least one grave error in one in ten.","excerpt":"One in ten papers contained at least one grave error","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"The post cautions that the two error-checking exercises are not directly comparable.","excerpt":"The two exercises are not directly comparable","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"A study cited in the post found that AI can help check a candidate flaw, but does not reliably replace human judgment in finding it.","excerpt":"“does not yet reliably substitute for the human judgment that locates the flaw in the first place”","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"A cited study found AI did better in crowded fields, while human performance was the same regardless of literature size.","excerpt":"AI did better in crowded fields with lots of existing research to digest. Humans performed at the same level regardless of the size of the literature.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"},{"text":"In a reproduction study, discrepancies were attributed roughly equally to problems with AI and human work, with better models shifting the balance against humans.","excerpt":"The discrepancies between the original papers and the AI versions were due in roughly equal measure to problems with the AI and human work. But the better the AI model, the more the balance tilted against humans.","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","checkedAt":"2026-11-20T10:16:10.161Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"As AI models improve, does the balance of discrepancies in reproducing social-science papers continue to shift against human work?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Marginal Revolution: AIs vs. humans in economics papers","summary":"Marginal Revolution’s post compares studies of AI and human work in economics and related social sciences. It reports that an AI verifier found severe errors in 842 of 1,000 APE papers, while a review of 100 papers accepted by leading economics journals flagged at least one grave error in one in ten. The post cautions that these exercises are not directly comparable. Its account is not a simple contest over error rates: one study found AI can lower the cost of checking a candidate flaw, but “does not yet reliably substitute for the human judgment that locates the flaw in the first place”. Another comparison found that the difference between APE and published papers was mostly in ideas, not execution. AI did better in crowded research fields, while human performance was the same regardless of literature size. In a separate reproduction exercise, discrepancies between AI and human versions of social-science papers were due in roughly equal measure to problems with each. But the better the AI model, the more the balance tilted against humans. The studies point to differences across research tasks, while leaving open how that balance may change as models improve.","question":"As AI models improve, does the balance of discrepancies in reproducing social-science papers continue to shift against human work?","claims":[{"text":"The post reports that an AI verifier found at least one severe error in 842 of 1,000 APE papers.","excerpt":"None was error-free; 842 contained at least one severe error, including ones invalidating headline results.","support":"source-stated"},{"text":"A review of a random sample of 100 papers accepted by leading economics journals flagged at least one grave error in one in ten.","excerpt":"One in ten papers contained at least one grave error","support":"source-stated"},{"text":"The post cautions that the two error-checking exercises are not directly comparable.","excerpt":"The two exercises are not directly comparable","support":"source-stated"},{"text":"A study cited in the post found that AI can help check a candidate flaw, but does not reliably replace human judgment in finding it.","excerpt":"“does not yet reliably substitute for the human judgment that locates the flaw in the first place”","support":"source-stated"},{"text":"A cited study found AI did better in crowded fields, while human performance was the same regardless of literature size.","excerpt":"AI did better in crowded fields with lots of existing research to digest. Humans performed at the same level regardless of the size of the literature.","support":"source-stated"},{"text":"In a reproduction study, discrepancies were attributed roughly equally to problems with AI and human work, with better models shifting the balance against humans.","excerpt":"The discrepancies between the original papers and the AI versions were due in roughly equal measure to problems with the AI and human work. But the better the AI model, the more the balance tilted against humans.","support":"source-stated"}],"limitations":["The post summarizes multiple studies but does not provide their full methods or enough detail to assess how the error categories were defined.","The post explicitly says the two error-checking exercises are not directly comparable; their reported error rates should not be treated as a head-to-head result.","The reproduction study excerpt does not specify how much better AI models changed the balance or whether the trend continues beyond the models studied."],"mediaId":"7a934f0174d68735bf149ec3cd47a05260a42c2c1e1b1d4a1dffd31c956247aa","correctionReason":null,"id":"4ccdc7a62f2032c232b637124169a98de568a28f1fa00923a789295a76b04d22","publicationId":"5bf85a5b-d449-421d-8a81-ad11ff24720c","version":1,"sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","sourceTitle":"AIs vs. humans in economics papers - Marginal REVOLUTION","publisher":"Marginal Revolution","publishedAt":"2026-11-20T05:53:10+00:00","originalPublished":{"value":"2026-11-20T05:53:10+00:00","precision":"instant","basis":"publisher_page"},"firstObservedAt":"2026-11-20T05:55:02.361Z","createdAt":"2026-11-20T06:01:45.012Z","historical":false,"entityIds":["marginal-revolution"],"topicIds":["power-and-infrastructure","work-and-agency"],"media":[{"id":"7a934f0174d68735bf149ec3cd47a05260a42c2c1e1b1d4a1dffd31c956247aa","kind":"article","url":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-20T06:00:51.422Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","checkedAt":"2026-11-20T10:16:10.161Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v9","mediaId":"7a934f0174d68735bf149ec3cd47a05260a42c2c1e1b1d4a1dffd31c956247aa","confidence":0.99,"provider":"typesafe","model":"jev-1.13.0","inputHash":"3397987c82c3d0e5bf509e2eee1517ab3414e37c58f4b80bbbf005daedd0390c","decidedAt":"2026-11-20T06:00:52.312Z"},"provenance":{"textHash":"01877528884b64ff985d1d1213bce53a6896a2f3de9f4753e846a2452f601cfd","observationId":"ce415bffa0e69325137c0324b24f8847d0729d68c4d32a4bba28c687b526ec07","retrievedAt":"2026-11-20T06:00:51.272Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"2f733214-3d9e-41fc-8a90-48095592b3d2"},"reviewer":{"model":"gpt-6-luna","runId":"90ea0ddd-1912-4ab6-86a8-8b3a8e206467"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-20T06:01:45.012Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Tyler Cowen","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Tyler Cowen","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Tyler Cowen","status":"stated","provenance":{"basis":"article-jsonld","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","locator":"document/html[1]/head[0]/script[53]@2221.@graph[0].author"},"assertionId":"010b14911ffe329db48c84e08ea18e89c284f49e29c13a6319939b5c8db68461:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-20T10:16:10.161Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"010b14911ffe329db48c84e08ea18e89c284f49e29c13a6319939b5c8db68461","sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","contentSha256":"2549a1d974cd6e8ae49934141835253d111365e04446c5cf10c1cfd1d74f6a77","textHash":"01877528884b64ff985d1d1213bce53a6896a2f3de9f4753e846a2452f601cfd","observedAt":"2026-11-20T10:16:10.161Z"}}],"participationCoverage":{"total":2,"returned":1,"complete":false,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://marginalrevolution.com/marginalrevolution/2026/11/ais-vs-humans-in-economics-papers.html","snapshotId":"010b14911ffe329db48c84e08ea18e89c284f49e29c13a6319939b5c8db68461","contentSha256":"2549a1d974cd6e8ae49934141835253d111365e04446c5cf10c1cfd1d74f6a77","textHash":"01877528884b64ff985d1d1213bce53a6896a2f3de9f4753e846a2452f601cfd","observedAt":"2026-11-20T10:16:10.161Z","basis":"publisher-page-metadata","sourceLinks":1,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/4ccdc7a62f2032c232b637124169a98de568a28f1fa00923a789295a76b04d22/source"}}}],"audits":[{"publicationId":"5bf85a5b-d449-421d-8a81-ad11ff24720c","verdict":"uncertain","findings":[{"dimension":"attribution","status":"uncertain","explanation":"The reading presents everything as Marginal Revolution's own post comparing studies, but the text is an excerpt from The Economist. The 100-paper review was run by The Economist with Refine, and the 'not directly comparable' caution is The Economist's wording, not MR's. The named researchers (Yanagizawa-Drott and Willner, Toda, Zampa, Kohler) are also omitted. Smallest repair: say MR is quoting The Economist, and credit the 100-paper review to The Economist and Refine.","excerpt":"Here is more from The Economist, interesting throughout."},{"dimension":"summary","status":"uncertain","explanation":"'APE papers' is the central term and is never explained. A general reader cannot tell that these are AI-generated (agentic research) papers, so the headline 842/1,000 comparison is hard to follow. The source's own framing is also flattened: it says humans' lower error rate 'might give them some hope' and that people retain the edge in ideas. Repair: briefly gloss APE as AI-agent-produced economics papers and keep that hedged conclusion.","excerpt":"Is agentic research any good?"},{"dimension":"date","status":"pass","explanation":"The originalPublished value, 2026-11-20T05:53:10Z, comes from the publisher page and matches the feed timestamp. No earlier publication is indicated, and the draft makes no conflicting date claim.","excerpt":null},{"dimension":"support","status":"uncertain","explanation":"The claims are grounded in the text, but several scope conditions are dropped. First, the crowded-fields result comes from an AI referee grading the papers, and the reading presents it without that condition. Second, Toda's finding applied to economic theory papers only. Third, the claim drops 'yet' from 'does not yet reliably substitute'. Repairs: note that an AI referee did the grading, restrict Toda's finding to theory papers, and keep 'yet'. The question is fairly derived from the reproduction finding.","excerpt":"Michele Zampa of the Geneva Graduate Institute used an AI referee to grade 981 AI papers and 41 human ones"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the original Marginal Revolution post. It is the most relevant available item.","excerpt":null}],"createdAt":"2026-11-20T06:01:45.012Z","model":"anthropic/claude-opus-5.5"},{"publicationId":"e92a98fe-9438-4789-80f8-3d17db77f4fa","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The draft correctly presents this as a Marginal Revolution post excerpting The Economist. It attributes the Refine review to The Economist and Refine, and each study finding to the cited researchers or studies, without presenting them as established fact.","excerpt":"Here is more from The Economist, interesting throughout."},{"dimension":"summary","status":"uncertain","explanation":"The central term APE is never explained, so a non-specialist cannot tell that the 842-of-1,000 figure concerns AI-generated agentic research papers. The comparison depends on this. Smallest repair: on first use, describe APE papers as AI-produced (agentic) research papers. The draft also drops the source's stated takeaway that humans' lower error rate offers some hope and that people retain the edge in ideas. Restoring that clause would preserve the source's framing.","excerpt":"The two exercises are not directly comparable, but human economists’ lower error rate might give them some hope. In ideas, too, people retain the edge."},{"dimension":"date","status":"pass","explanation":"The originalPublished value of 2026-11-20T05:53:10Z comes from the publisher page and matches the feed entry for this Marginal Revolution post. The draft makes no conflicting date claim.","excerpt":null},{"dimension":"support","status":"pass","explanation":"All claims match the excerpted text, including the figures, quotations and the non-comparability caveat. The phrase 'as the AI model improved' slightly reframes 'the better the AI model', but it stays within the source's meaning. The question is properly framed as unresolved, and the limitations correctly note the missing detail.","excerpt":"But the better the AI model, the more the balance tilted against humans."},{"dimension":"media","status":"pass","explanation":"The selected media ID is the verified direct link to the original Marginal Revolution post, which is the source itself.","excerpt":null}],"createdAt":"2026-11-20T13:02:12.337Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}