{"id":"71a96380ad1076f5039a55dff63cecc978e936d1ad838e663bc18cc5346cb6e9","title":"Athena Chapekis and coauthors argue in Pew Research Center report that synthetic polls fall short","argument":"Pew Research Center’s report, published September 30, 2026, examines whether AI-generated survey data can reproduce results from high-quality polls of people. Athena Chapekis, Arnold Lau, Samuel Bestvater, Sono Shah, Andrew Mercer and Aaron Smith report that AI respondents did not match human respondents especially well across three replicated American Trends Panel survey waves. They say synthetic results differed from human estimates by an average of 12 percentage points, with errors across all major subgroups; performance was especially poor for several groups, including Republicans and Republican leaners and Black adults.\n\nThe authors also point to cases where simulated responses overstated Hispanic adults’ interest in the World Cup and made views within political groups seem more uniform than they were in human surveys. They connect such results to concerns that models can rely on assumptions or stereotypes about groups. The report says the experiment was designed to evaluate synthetic samples for public-opinion research, not other uses of AI in polling. The Center says speaking to the public is essential to measuring public opinion and that it has no current or future plans to use AI models to generate survey results. Whether other models or approaches would produce similar errors remains an open question.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 30, 2026; originally published September 30, 2026. The report’s main findings here concern three replicated American Trends Panel survey waves; synthetic results generally use Claude Opus 4.6. The evidence does not establish how other models or sampling approaches would perform.","speaker":{"name":"Pew Research Center","roleAtSource":"Publisher"},"source":{"title":"How well synthetic samples replicate public opinion","publisher":"Pew Research Center","format":"Article","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","publishedAt":"2026-09-30T17:55:18.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"The report is dated September 30, 2026.","excerpt":"Report\n|\nSeptember 30, 2026","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","relationship":"Source-stated"},{"text":"The authors report that synthetic polling results differed from human poll estimates by an average of 12 percentage points across the three waves.","excerpt":"Across all three waves, the synthetic polling results differed by an average of 12 percentage points from the estimates produced by the human polls.","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","relationship":"Source-stated"},{"text":"The report says errors appeared across all major subgroups and were especially high for several groups, including Republicans and Republican leaners and Black adults.","excerpt":"There were also clear errors for all major subgroups in the analysis.","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","relationship":"Source-stated"},{"text":"The authors report examples in which synthetic results overstated Hispanic adults’ interest in the World Cup and made political views seem more uniform than in the actual population.","excerpt":"Across numerous such examples, our AI respondents were far less politically diverse than the actual population.","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","relationship":"Source-stated"},{"text":"The Center says speaking to the public is essential to measuring public opinion and that it has no current or future plans to use AI models to generate survey results.","excerpt":"The Center believes that speaking to the public is essential to measuring public opinion and has no current or future plans to use AI models to generate survey results.","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How would the reported errors change with other AI models or different ways of constructing synthetic samples?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Athena Chapekis and coauthors argue in Pew Research Center report that synthetic polls fall short","summary":"Pew Research Center’s report, published September 30, 2026, examines whether AI-generated survey data can reproduce results from high-quality polls of people. Athena Chapekis, Arnold Lau, Samuel Bestvater, Sono Shah, Andrew Mercer and Aaron Smith report that AI respondents did not match human respondents especially well across three replicated American Trends Panel survey waves. They say synthetic results differed from human estimates by an average of 12 percentage points, with errors across all major subgroups; performance was especially poor for several groups, including Republicans and Republican leaners and Black adults.\n\nThe authors also point to cases where simulated responses overstated Hispanic adults’ interest in the World Cup and made views within political groups seem more uniform than they were in human surveys. They connect such results to concerns that models can rely on assumptions or stereotypes about groups. The report says the experiment was designed to evaluate synthetic samples for public-opinion research, not other uses of AI in polling. The Center says speaking to the public is essential to measuring public opinion and that it has no current or future plans to use AI models to generate survey results. Whether other models or approaches would produce similar errors remains an open question.","question":"How would the reported errors change with other AI models or different ways of constructing synthetic samples?","claims":[{"text":"The report is dated September 30, 2026.","excerpt":"Report\n|\nSeptember 30, 2026","support":"source-stated"},{"text":"The authors report that synthetic polling results differed from human poll estimates by an average of 12 percentage points across the three waves.","excerpt":"Across all three waves, the synthetic polling results differed by an average of 12 percentage points from the estimates produced by the human polls.","support":"source-stated"},{"text":"The report says errors appeared across all major subgroups and were especially high for several groups, including Republicans and Republican leaners and Black adults.","excerpt":"There were also clear errors for all major subgroups in the analysis.","support":"source-stated"},{"text":"The authors report examples in which synthetic results overstated Hispanic adults’ interest in the World Cup and made political views seem more uniform than in the actual population.","excerpt":"Across numerous such examples, our AI respondents were far less politically diverse than the actual population.","support":"source-stated"},{"text":"The Center says speaking to the public is essential to measuring public opinion and that it has no current or future plans to use AI models to generate survey results.","excerpt":"The Center believes that speaking to the public is essential to measuring public opinion and has no current or future plans to use AI models to generate survey results.","support":"source-stated"}],"limitations":["Text as retrieved on October 30, 2026; originally published September 30, 2026.","The report’s main findings here concern three replicated American Trends Panel survey waves; synthetic results generally use Claude Opus 4.6. The evidence does not establish how other models or sampling approaches would perform."],"mediaId":null,"correctionReason":null,"id":"71a96380ad1076f5039a55dff63cecc978e936d1ad838e663bc18cc5346cb6e9","publicationId":"92d76bd3-06e6-0a4a-2b82-dcd699e8e5d6","version":1,"sourceUrl":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","sourceTitle":"How well synthetic samples replicate public opinion","publisher":"Pew Research Center","publishedAt":"2026-09-30T17:55:18.000Z","originalPublished":{"value":"2026-09-30T17:55:18.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T03:43:21.510Z","createdAt":"2026-11-05T10:08:23.136Z","historical":true,"entityIds":["pew-research-center"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"4cd59c8ae0a389f5c16c323913934f992d0b5afd48743521c3e5f3aac0159277","rubricVersion":"em-backfill-context/3","provider":"typesafe","model":"jev-1.13.0","reasonCodes":["bridge_admitted_by_model"],"confidence":0.55,"decidedAt":"2026-10-30T04:57:55.697Z"},"provenance":{"textHash":"5060fc128b1d51cb24264b1434b1888602e7f1d00c4178de626a7ecfe1e5b48c","observationId":"f0b37a2f9329ca2e3c7e22a7b8b0caf0881f4412cd7c36e72457807bc7805225","retrievedAt":"2026-10-30T04:59:47.038Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"14e2e1d9-6371-776a-6c18-6d72d150547c"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"a8e1af5e-2086-f677-099a-580160616d68"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-05T10:03:45.542Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/athena-chapekis-and-coauthors-argue-in-pew-research-center-71a96380","formats":{"html":"/readings/athena-chapekis-and-coauthors-argue-in-pew-research-center-71a96380","markdown":"/readings/athena-chapekis-and-coauthors-argue-in-pew-research-center-71a96380.md","json":"/readings/athena-chapekis-and-coauthors-argue-in-pew-research-center-71a96380.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[],"entities":[{"id":"pew-research-center","name":"Pew Research Center","kind":"organization","context":"Pew Research Center studies public attitudes and social change through surveys and other empirical research.","referenceUrls":["https://www.pewresearch.org/","https://www.pewresearch.org/internet/2025/04/03/how-the-us-public-and-ai-experts-view-artificial-intelligence/"],"path":"/entities/pew-research-center","json":"/entities/pew-research-center.json","markdown":"/entities/pew-research-center.md"},{"id":"claude","name":"Claude","kind":"model","context":"Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.","referenceUrls":["https://claude.com/product/overview","https://www.anthropic.com/news","https://www.anthropic.com/research/team/economics"],"path":"/entities/claude","json":"/entities/claude.json","markdown":"/entities/claude.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"71a96380ad1076f5039a55dff63cecc978e936d1ad838e663bc18cc5346cb6e9","revision":"92d76bd3-06e6-0a4a-2b82-dcd699e8e5d6","revisionKind":"publication-id","title":"Athena Chapekis and coauthors argue in Pew Research Center report that synthetic polls fall short","url":"https://escapemeridian.com/readings/athena-chapekis-and-coauthors-argue-in-pew-research-center-71a96380"},"source":{"type":"source","id":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","revision":"5060fc128b1d51cb24264b1434b1888602e7f1d00c4178de626a7ecfe1e5b48c","revisionKind":"sha256-source-text","title":"How well synthetic samples replicate public opinion","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"71a96380ad1076f5039a55dff63cecc978e936d1ad838e663bc18cc5346cb6e9","title":"Athena Chapekis and coauthors argue in Pew Research Center report that synthetic polls fall short","argument":"Pew Research Center’s report, published September 30, 2026, examines whether AI-generated survey data can reproduce results from high-quality polls of people. Athena Chapekis, Arnold Lau, Samuel Bestvater, Sono Shah, Andrew Mercer and Aaron Smith report that AI respondents did not match human respondents especially well across three replicated American Trends Panel survey waves. They say synthetic results differed from human estimates by an average of 12 percentage points, with errors across all major subgroups; performance was especially poor for several groups, including Republicans and Republican leaners and Black adults.\n\nThe authors also point to cases where simulated responses overstated Hispanic adults’ interest in the World Cup and made views within political groups seem more uniform than they were in human surveys. They connect such results to concerns that models can rely on assumptions or stereotypes about groups. The report says the experiment was designed to evaluate synthetic samples for public-opinion research, not other uses of AI in polling. The Center says speaking to the public is essential to measuring public opinion and that it has no current or future plans to use AI models to generate survey results. Whether other models or approaches would produce similar errors remains an open question.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 30, 2026; originally published September 30, 2026. The report’s main findings here concern three replicated American Trends Panel survey waves; synthetic results generally use Claude Opus 4.6. The evidence does not establish how other models or sampling approaches would perform.","speaker":{"name":"Pew Research Center","roleAtSource":"Publisher"},"source":{"title":"How well synthetic samples replicate public opinion","publisher":"Pew Research Center","format":"Article","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","publishedAt":"2026-09-30T17:55:18.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"The report is dated September 30, 2026.","excerpt":"Report\n|\nSeptember 30, 2026","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","relationship":"Source-stated"},{"text":"The authors report that synthetic polling results differed from human poll estimates by an average of 12 percentage points across the three waves.","excerpt":"Across all three waves, the synthetic polling results differed by an average of 12 percentage points from the estimates produced by the human polls.","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","relationship":"Source-stated"},{"text":"The report says errors appeared across all major subgroups and were especially high for several groups, including Republicans and Republican leaners and Black adults.","excerpt":"There were also clear errors for all major subgroups in the analysis.","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","relationship":"Source-stated"},{"text":"The authors report examples in which synthetic results overstated Hispanic adults’ interest in the World Cup and made political views seem more uniform than in the actual population.","excerpt":"Across numerous such examples, our AI respondents were far less politically diverse than the actual population.","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","relationship":"Source-stated"},{"text":"The Center says speaking to the public is essential to measuring public opinion and that it has no current or future plans to use AI models to generate survey results.","excerpt":"The Center believes that speaking to the public is essential to measuring public opinion and has no current or future plans to use AI models to generate survey results.","url":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How would the reported errors change with other AI models or different ways of constructing synthetic samples?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Athena Chapekis and coauthors argue in Pew Research Center report that synthetic polls fall short","summary":"Pew Research Center’s report, published September 30, 2026, examines whether AI-generated survey data can reproduce results from high-quality polls of people. Athena Chapekis, Arnold Lau, Samuel Bestvater, Sono Shah, Andrew Mercer and Aaron Smith report that AI respondents did not match human respondents especially well across three replicated American Trends Panel survey waves. They say synthetic results differed from human estimates by an average of 12 percentage points, with errors across all major subgroups; performance was especially poor for several groups, including Republicans and Republican leaners and Black adults.\n\nThe authors also point to cases where simulated responses overstated Hispanic adults’ interest in the World Cup and made views within political groups seem more uniform than they were in human surveys. They connect such results to concerns that models can rely on assumptions or stereotypes about groups. The report says the experiment was designed to evaluate synthetic samples for public-opinion research, not other uses of AI in polling. The Center says speaking to the public is essential to measuring public opinion and that it has no current or future plans to use AI models to generate survey results. Whether other models or approaches would produce similar errors remains an open question.","question":"How would the reported errors change with other AI models or different ways of constructing synthetic samples?","claims":[{"text":"The report is dated September 30, 2026.","excerpt":"Report\n|\nSeptember 30, 2026","support":"source-stated"},{"text":"The authors report that synthetic polling results differed from human poll estimates by an average of 12 percentage points across the three waves.","excerpt":"Across all three waves, the synthetic polling results differed by an average of 12 percentage points from the estimates produced by the human polls.","support":"source-stated"},{"text":"The report says errors appeared across all major subgroups and were especially high for several groups, including Republicans and Republican leaners and Black adults.","excerpt":"There were also clear errors for all major subgroups in the analysis.","support":"source-stated"},{"text":"The authors report examples in which synthetic results overstated Hispanic adults’ interest in the World Cup and made political views seem more uniform than in the actual population.","excerpt":"Across numerous such examples, our AI respondents were far less politically diverse than the actual population.","support":"source-stated"},{"text":"The Center says speaking to the public is essential to measuring public opinion and that it has no current or future plans to use AI models to generate survey results.","excerpt":"The Center believes that speaking to the public is essential to measuring public opinion and has no current or future plans to use AI models to generate survey results.","support":"source-stated"}],"limitations":["Text as retrieved on October 30, 2026; originally published September 30, 2026.","The report’s main findings here concern three replicated American Trends Panel survey waves; synthetic results generally use Claude Opus 4.6. The evidence does not establish how other models or sampling approaches would perform."],"mediaId":null,"correctionReason":null,"id":"71a96380ad1076f5039a55dff63cecc978e936d1ad838e663bc18cc5346cb6e9","publicationId":"92d76bd3-06e6-0a4a-2b82-dcd699e8e5d6","version":1,"sourceUrl":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","sourceTitle":"How well synthetic samples replicate public opinion","publisher":"Pew Research Center","publishedAt":"2026-09-30T17:55:18.000Z","originalPublished":{"value":"2026-09-30T17:55:18.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T03:43:21.510Z","createdAt":"2026-11-05T10:08:23.136Z","historical":true,"entityIds":["pew-research-center"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-samples-replicate-public-opinion/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"4cd59c8ae0a389f5c16c323913934f992d0b5afd48743521c3e5f3aac0159277","rubricVersion":"em-backfill-context/3","provider":"typesafe","model":"jev-1.13.0","reasonCodes":["bridge_admitted_by_model"],"confidence":0.55,"decidedAt":"2026-10-30T04:57:55.697Z"},"provenance":{"textHash":"5060fc128b1d51cb24264b1434b1888602e7f1d00c4178de626a7ecfe1e5b48c","observationId":"f0b37a2f9329ca2e3c7e22a7b8b0caf0881f4412cd7c36e72457807bc7805225","retrievedAt":"2026-10-30T04:59:47.038Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"14e2e1d9-6371-776a-6c18-6d72d150547c"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"a8e1af5e-2086-f677-099a-580160616d68"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-05T10:03:45.542Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"92d76bd3-06e6-0a4a-2b82-dcd699e8e5d6","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading correctly attributes the report to Pew Research Center and its six listed authors. It keeps findings attributed to the authors and states the Center's AI policy as the Center's own position.","excerpt":"By Athena Chapekis, Arnold Lau, Samuel Bestvater, Sono Shah, Andrew Mercer and Aaron Smith"},{"dimension":"summary","status":"uncertain","explanation":"The core findings are faithful and readable. Two gaps remain. First, the summary never explains how the AI respondents were built: an AI model adopted personas of real panelists, the 'digital twins' approach. It names neither the model, Claude Opus 4.6, nor that approach outside the limitations, so a lay reader cannot see what was tested. Second, the closing open question about other models leaves out that Pew tested several models and Opus 4.6 performed best. Smallest repair: add one sentence on the persona method and the model, and narrow the question to untested approaches.","excerpt":"We used a “digital twins” approach for this study, asking an AI model to adopt the personas of real humans who are members of the ATP."},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-09-30T17:55Z, so the UTC day is September 30, 2026. This matches the report's dateline, and the reading states the date at day precision. It uses no later events and no hindsight framing. The page is a publisher page, so no archive disclosure is needed, and modifiedAt equals the publication time.","excerpt":"Report\n|\nSeptember 30, 2026"},{"dimension":"support","status":"uncertain","explanation":"The stated claims are each supported by source excerpts: the 12-point average error, subgroup errors, the World Cup and partisan uniformity examples, and the Center's policy. However, the reading presents other models' performance as an open question, and a limitation says the evidence does not establish how other models would perform. The source says several models were evaluated and Opus 4.6 performed best, which implies the others did worse. Smallest repair: state that Opus 4.6 was the best of several models Pew tested, and limit the open question to approaches Pew did not test.","excerpt":"We tested multiple models, but all synthetic results unless noted otherwise come from Anthropic’s Claude Opus 4.6."},{"dimension":"media","status":"pass","explanation":"No media was selected, and no verified media candidates were supplied, so there is no media identity or relevance issue.","excerpt":null}],"createdAt":"2026-11-05T10:03:45.542Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}