Pew Research Center reports model choice reshapes synthetic polling results

Pew Research Center reports model choice reshapes synthetic polling results

Pew Research Center’s report, published September 30, 2026, compares AI-generated survey responses with human poll results and examines how results vary by model. The Center used GPT-5.1 and Claude Opus 4.6 to simulate people from its American Trends Panel, using the same questionnaire and respondent profiles. It reports that neither model reproduced the human panel especially well, and that the models often produced sharply different pictures of public opinion. For example, GPT tended to favor extreme answers, while Opus tended to favor middle options; the report says this pattern appeared across questions on different topics. The difference is more than a matter of which model has the lower average error: choosing a model can change the story a synthetic poll tells about issues and groups. That matters because readers of such polls may encounter incompatible depictions of public attitudes even when the synthetic samples are built in the same way. The Center says newer models and changes to synthetic sample construction could reduce errors or alter these patterns. Its findings concern this experiment, including selected panelists and surveys, and do not address other uses of AI models in polling. How would these results change with other models, respondent profiles or survey questions?

Publication: 9b903ec1-c646-4563-9ef9-76e7205e20dc

attribution — pass: The reading attributes the findings to Pew Research Center, which matches the page's publisher and author byline. The page is one chapter of a single report with its own analysis, not a listing of other pieces. Claims are consistently framed as what Pew reports or says.

By Athena Chapekis, Arnold Lau, Samuel Bestvater, Sono Shah, Andrew Mercer and Aaron Smith

summary — pass: The reading leads with the actor, Pew, and plainly explains synthetic polling as AI-generated survey responses that simulate panel members. It identifies the two models and preserves the core thesis: model choice can change the story a synthetic poll tells. It accurately conveys the extreme-versus-middle response pattern and Pew's caveats. The question follows from the stated limits. The American Trends Panel is lightly explained but understandable from context. Minor gap: the page is one chapter of the larger report, but this does not mislead.

Here, the choice of model alone has the potential to reshape the narrative around public opinion of abortion.

date — pass: The originalPublished instant, 2026-09-30, matches the page dateline of September 30, 2026, and the reading states the date at day precision. The page is a publisher page, so no archive disclosure is needed. modifiedAt equals the publication time. The reading makes no hindsight claims and presents the possibility of future model improvements as Pew's conditional statement, not as an outcome.

September 30, 2026

support — pass: Each claim matches the source: the error figures (11.4 vs 13.3 points); the very different pictures of public opinion; GPT favoring extreme options and Opus favoring middle ones regardless of topic; and model choice alone producing systematic differences. The inference about readers encountering incompatible depictions follows directly from Pew's own narrative framing. The caveats about newer models and scope are source-stated.

This was broadly true regardless of what the scale was or what the question was about.

media — pass: No media was selected, and no verified media candidates were supplied. The source page's media status is unchecked with no items, so omitting media is appropriate.