{"id":"1bf136d6d812e1f8483beb925af6ec2894a55984b7f5af170197a98cbbb97a4d","title":"Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations","argument":"Authors Sion Weatherhead, Ben Newell, Aaron Belbasis and Flora Salim argue that LLM agents can reproduce some human trait distributions in questionnaires without reproducing the differences in reasoning and behaviour those traits are meant to represent. They compare simulated agents conditioned on synthetic or human-derived profiles with a cohort of 473 adults, using cognitive tests, trait questionnaires and decision scenarios. Across seven simulation runs of four models, agents matched Need for Cognition and Big Five distributions on self-report questionnaires, but often performed unlike people on the cognitive tasks. In four Claude Sonnet 4.6 runs, every agent earned full marks on both CRTs regardless of assigned reflection level; in the other runs, full-mark rates still far exceeded the human sample. The BRT scenarios also showed overproduction of many coded behaviours, while links between reflection and behaviour carried over only in part. The authors’ point is that assigning a persona or matching its questionnaire scores is not enough: a useful synthetic population must also preserve the structured variation in reasoning, errors and behaviour associated with those traits. These results describe the tested models and tasks; they do not establish how well other simulations or research settings will perform.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract reports results from seven full-battery simulation runs of four models; it does not identify all four models by name. The abstract describes performance on the stated tests and scenarios; it does not establish how the results generalize to other research settings.","speaker":{"name":"Weatherhead, Sion, Newell, Ben, Belbasis, Aaron, Salim, Flora","roleAtSource":"Article author"},"source":{"title":"Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.10508","publishedAt":"2026-11-07","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors evaluate whether simulated populations reproduce human variation in reasoning, traits, associations and downstream behaviour.","excerpt":"tests whether simulated populations reproduce human variation in reasoning, self-reported traits, their associations, and downstream behaviour.","url":"https://arxiv.org/abs/2611.10508","relationship":"Source-stated"},{"text":"Across seven simulation runs of four models, agents recovered Need for Cognition and Big Five distributions on questionnaires but fell short on cognitive-task performance and downstream behaviours.","excerpt":"Across seven full-battery simulation runs of four models, agents recover Need for Cognition and Big Five trait distributions on self-report questionnaires, but fall short on simulating diversity of cognitive task performance and downstream behaviours.","url":"https://arxiv.org/abs/2611.10508","relationship":"Source-stated"},{"text":"In the four Claude Sonnet 4.6 runs, every agent scored full marks on both CRTs regardless of assigned reflection level.","excerpt":"In the four Claude Sonnet 4.6 runs, every agent scores full marks on both CRTs, whatever reflection level its persona was assigned.","url":"https://arxiv.org/abs/2611.10508","relationship":"Source-stated"},{"text":"Across the BRT scenarios, every run over-produced many coded behaviours, and human links between reflection and behaviour carried over only in part.","excerpt":"In the BRT scenarios, every run over-produces 8 to 11 of 15 coded behaviours by at least 20 percentage points, and people's links between reflection and behaviour carry over only in part.","url":"https://arxiv.org/abs/2611.10508","relationship":"Source-stated"},{"text":"The authors argue that useful synthetic populations must preserve structured variation in reasoning, errors and behaviour, not only trait descriptors.","excerpt":"Useful synthetic populations must preserve not only trait descriptors but the structured variation in reasoning, error type, and behaviour those descriptors are meant to organise.","url":"https://arxiv.org/abs/2611.10508","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.10508","checkedAt":"2026-11-12T12:50:09.938Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What approach would allow simulated populations to reproduce the human variation in reasoning, error type and behaviour that the authors say their trait profiles are meant to organise?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations","summary":"Authors Sion Weatherhead, Ben Newell, Aaron Belbasis and Flora Salim argue that LLM agents can reproduce some human trait distributions in questionnaires without reproducing the differences in reasoning and behaviour those traits are meant to represent. They compare simulated agents conditioned on synthetic or human-derived profiles with a cohort of 473 adults, using cognitive tests, trait questionnaires and decision scenarios. Across seven simulation runs of four models, agents matched Need for Cognition and Big Five distributions on self-report questionnaires, but often performed unlike people on the cognitive tasks. In four Claude Sonnet 4.6 runs, every agent earned full marks on both CRTs regardless of assigned reflection level; in the other runs, full-mark rates still far exceeded the human sample. The BRT scenarios also showed overproduction of many coded behaviours, while links between reflection and behaviour carried over only in part. The authors’ point is that assigning a persona or matching its questionnaire scores is not enough: a useful synthetic population must also preserve the structured variation in reasoning, errors and behaviour associated with those traits. These results describe the tested models and tasks; they do not establish how well other simulations or research settings will perform.","question":"What approach would allow simulated populations to reproduce the human variation in reasoning, error type and behaviour that the authors say their trait profiles are meant to organise?","claims":[{"text":"The authors evaluate whether simulated populations reproduce human variation in reasoning, traits, associations and downstream behaviour.","excerpt":"tests whether simulated populations reproduce human variation in reasoning, self-reported traits, their associations, and downstream behaviour.","support":"source-stated"},{"text":"Across seven simulation runs of four models, agents recovered Need for Cognition and Big Five distributions on questionnaires but fell short on cognitive-task performance and downstream behaviours.","excerpt":"Across seven full-battery simulation runs of four models, agents recover Need for Cognition and Big Five trait distributions on self-report questionnaires, but fall short on simulating diversity of cognitive task performance and downstream behaviours.","support":"source-stated"},{"text":"In the four Claude Sonnet 4.6 runs, every agent scored full marks on both CRTs regardless of assigned reflection level.","excerpt":"In the four Claude Sonnet 4.6 runs, every agent scores full marks on both CRTs, whatever reflection level its persona was assigned.","support":"source-stated"},{"text":"Across the BRT scenarios, every run over-produced many coded behaviours, and human links between reflection and behaviour carried over only in part.","excerpt":"In the BRT scenarios, every run over-produces 8 to 11 of 15 coded behaviours by at least 20 percentage points, and people's links between reflection and behaviour carry over only in part.","support":"source-stated"},{"text":"The authors argue that useful synthetic populations must preserve structured variation in reasoning, errors and behaviour, not only trait descriptors.","excerpt":"Useful synthetic populations must preserve not only trait descriptors but the structured variation in reasoning, error type, and behaviour those descriptors are meant to organise.","support":"source-stated"}],"limitations":["The abstract reports results from seven full-battery simulation runs of four models; it does not identify all four models by name.","The abstract describes performance on the stated tests and scenarios; it does not establish how the results generalize to other research settings."],"mediaId":"ce51cf9ee6541848e0f1d4f102a72b379a1fc239d6effe42ef6c62f4d4302800","correctionReason":null,"id":"1bf136d6d812e1f8483beb925af6ec2894a55984b7f5af170197a98cbbb97a4d","publicationId":"fef6eea9-f02f-4206-8fe5-89b14e0c761b","version":1,"sourceUrl":"https://arxiv.org/abs/2611.10508","sourceTitle":"Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-07","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-10T08:39:26.779Z","createdAt":"2026-11-12T11:46:57.286Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"ce51cf9ee6541848e0f1d4f102a72b379a1fc239d6effe42ef6c62f4d4302800","kind":"article","url":"https://arxiv.org/abs/2611.10508","sourceUrl":"https://arxiv.org/abs/2611.10508","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-12T11:34:10.767Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.10508","checkedAt":"2026-11-12T12:50:09.938Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v6","mediaId":"ce51cf9ee6541848e0f1d4f102a72b379a1fc239d6effe42ef6c62f4d4302800","confidence":0.8500000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"c032ba4e160501247c60f006bd7593d5e1a1fb2b1ef50e1727f48ae179bf4b4e","decidedAt":"2026-11-12T11:34:13.197Z"},"provenance":{"textHash":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","observationId":"718d1333fa8d938281c0bfd86e721cffa6e88861801b4d980c488713c301f5c9","retrievedAt":"2026-11-12T11:34:10.431Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"d364baf3-3ed3-4b51-87cd-e5848392495e"},"reviewer":{"model":"gpt-6-luna","runId":"577d1aa6-2f60-4d62-88f2-e82bdd94ba1e"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-12T11:46:57.286Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Weatherhead, Sion","url":null,"kind":"unknown"},{"name":"Newell, Ben","url":null,"kind":"unknown"},{"name":"Belbasis, Aaron","url":null,"kind":"unknown"},{"name":"Salim, Flora","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Weatherhead, Sion","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Weatherhead, Sion","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10508","locator":"head/meta[name=author|citation_author]"},"assertionId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T12:50:09.938Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9","sourceUrl":"https://arxiv.org/abs/2611.10508","contentSha256":"93f6ba196f68dd1ddbf71852cf2940262b820a53a36697b7c645cc153cee4d08","textHash":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","observedAt":"2026-11-12T12:50:09.938Z"}},{"actor":{"name":"Newell, Ben","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Newell, Ben","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10508","locator":"head/meta[name=author|citation_author]"},"assertionId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T12:50:09.938Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9","sourceUrl":"https://arxiv.org/abs/2611.10508","contentSha256":"93f6ba196f68dd1ddbf71852cf2940262b820a53a36697b7c645cc153cee4d08","textHash":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","observedAt":"2026-11-12T12:50:09.938Z"}},{"actor":{"name":"Belbasis, Aaron","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Belbasis, Aaron","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10508","locator":"head/meta[name=author|citation_author]"},"assertionId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T12:50:09.938Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9","sourceUrl":"https://arxiv.org/abs/2611.10508","contentSha256":"93f6ba196f68dd1ddbf71852cf2940262b820a53a36697b7c645cc153cee4d08","textHash":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","observedAt":"2026-11-12T12:50:09.938Z"}},{"actor":{"name":"Salim, Flora","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Salim, Flora","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10508","locator":"head/meta[name=author|citation_author]"},"assertionId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T12:50:09.938Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9","sourceUrl":"https://arxiv.org/abs/2611.10508","contentSha256":"93f6ba196f68dd1ddbf71852cf2940262b820a53a36697b7c645cc153cee4d08","textHash":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","observedAt":"2026-11-12T12:50:09.938Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.10508","snapshotId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9","contentSha256":"93f6ba196f68dd1ddbf71852cf2940262b820a53a36697b7c645cc153cee4d08","textHash":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","observedAt":"2026-11-12T12:50:09.938Z","basis":"publisher-page-metadata","sourceLinks":44,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/1bf136d6d812e1f8483beb925af6ec2894a55984b7f5af170197a98cbbb97a4d/source"}},"canonicalUrl":"https://escapemeridian.com/readings/imperfectly-imperfect-evaluating-graded-cognitive-diversity-in-1bf136d6","formats":{"html":"/readings/imperfectly-imperfect-evaluating-graded-cognitive-diversity-in-1bf136d6","markdown":"/readings/imperfectly-imperfect-evaluating-graded-cognitive-diversity-in-1bf136d6.md","json":"/readings/imperfectly-imperfect-evaluating-graded-cognitive-diversity-in-1bf136d6.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"},{"id":"claude","name":"Claude","kind":"model","context":"Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.","referenceUrls":["https://claude.com/product/overview","https://www.anthropic.com/news","https://www.anthropic.com/research/team/economics"],"path":"/entities/claude","json":"/entities/claude.json","markdown":"/entities/claude.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"1bf136d6d812e1f8483beb925af6ec2894a55984b7f5af170197a98cbbb97a4d","revision":"fef6eea9-f02f-4206-8fe5-89b14e0c761b","revisionKind":"publication-id","title":"Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations","url":"https://escapemeridian.com/readings/imperfectly-imperfect-evaluating-graded-cognitive-diversity-in-1bf136d6"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.10508","revision":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","revisionKind":"sha256-source-text","title":"Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations","url":"https://arxiv.org/abs/2611.10508"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"1bf136d6d812e1f8483beb925af6ec2894a55984b7f5af170197a98cbbb97a4d","title":"Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations","argument":"Authors Sion Weatherhead, Ben Newell, Aaron Belbasis and Flora Salim argue that LLM agents can reproduce some human trait distributions in questionnaires without reproducing the differences in reasoning and behaviour those traits are meant to represent. They compare simulated agents conditioned on synthetic or human-derived profiles with a cohort of 473 adults, using cognitive tests, trait questionnaires and decision scenarios. Across seven simulation runs of four models, agents matched Need for Cognition and Big Five distributions on self-report questionnaires, but often performed unlike people on the cognitive tasks. In four Claude Sonnet 4.6 runs, every agent earned full marks on both CRTs regardless of assigned reflection level; in the other runs, full-mark rates still far exceeded the human sample. The BRT scenarios also showed overproduction of many coded behaviours, while links between reflection and behaviour carried over only in part. The authors’ point is that assigning a persona or matching its questionnaire scores is not enough: a useful synthetic population must also preserve the structured variation in reasoning, errors and behaviour associated with those traits. These results describe the tested models and tasks; they do not establish how well other simulations or research settings will perform.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract reports results from seven full-battery simulation runs of four models; it does not identify all four models by name. The abstract describes performance on the stated tests and scenarios; it does not establish how the results generalize to other research settings.","speaker":{"name":"Weatherhead, Sion, Newell, Ben, Belbasis, Aaron, Salim, Flora","roleAtSource":"Article author"},"source":{"title":"Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.10508","publishedAt":"2026-11-07","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors evaluate whether simulated populations reproduce human variation in reasoning, traits, associations and downstream behaviour.","excerpt":"tests whether simulated populations reproduce human variation in reasoning, self-reported traits, their associations, and downstream behaviour.","url":"https://arxiv.org/abs/2611.10508","relationship":"Source-stated"},{"text":"Across seven simulation runs of four models, agents recovered Need for Cognition and Big Five distributions on questionnaires but fell short on cognitive-task performance and downstream behaviours.","excerpt":"Across seven full-battery simulation runs of four models, agents recover Need for Cognition and Big Five trait distributions on self-report questionnaires, but fall short on simulating diversity of cognitive task performance and downstream behaviours.","url":"https://arxiv.org/abs/2611.10508","relationship":"Source-stated"},{"text":"In the four Claude Sonnet 4.6 runs, every agent scored full marks on both CRTs regardless of assigned reflection level.","excerpt":"In the four Claude Sonnet 4.6 runs, every agent scores full marks on both CRTs, whatever reflection level its persona was assigned.","url":"https://arxiv.org/abs/2611.10508","relationship":"Source-stated"},{"text":"Across the BRT scenarios, every run over-produced many coded behaviours, and human links between reflection and behaviour carried over only in part.","excerpt":"In the BRT scenarios, every run over-produces 8 to 11 of 15 coded behaviours by at least 20 percentage points, and people's links between reflection and behaviour carry over only in part.","url":"https://arxiv.org/abs/2611.10508","relationship":"Source-stated"},{"text":"The authors argue that useful synthetic populations must preserve structured variation in reasoning, errors and behaviour, not only trait descriptors.","excerpt":"Useful synthetic populations must preserve not only trait descriptors but the structured variation in reasoning, error type, and behaviour those descriptors are meant to organise.","url":"https://arxiv.org/abs/2611.10508","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.10508","checkedAt":"2026-11-12T12:50:09.938Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What approach would allow simulated populations to reproduce the human variation in reasoning, error type and behaviour that the authors say their trait profiles are meant to organise?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations","summary":"Authors Sion Weatherhead, Ben Newell, Aaron Belbasis and Flora Salim argue that LLM agents can reproduce some human trait distributions in questionnaires without reproducing the differences in reasoning and behaviour those traits are meant to represent. They compare simulated agents conditioned on synthetic or human-derived profiles with a cohort of 473 adults, using cognitive tests, trait questionnaires and decision scenarios. Across seven simulation runs of four models, agents matched Need for Cognition and Big Five distributions on self-report questionnaires, but often performed unlike people on the cognitive tasks. In four Claude Sonnet 4.6 runs, every agent earned full marks on both CRTs regardless of assigned reflection level; in the other runs, full-mark rates still far exceeded the human sample. The BRT scenarios also showed overproduction of many coded behaviours, while links between reflection and behaviour carried over only in part. The authors’ point is that assigning a persona or matching its questionnaire scores is not enough: a useful synthetic population must also preserve the structured variation in reasoning, errors and behaviour associated with those traits. These results describe the tested models and tasks; they do not establish how well other simulations or research settings will perform.","question":"What approach would allow simulated populations to reproduce the human variation in reasoning, error type and behaviour that the authors say their trait profiles are meant to organise?","claims":[{"text":"The authors evaluate whether simulated populations reproduce human variation in reasoning, traits, associations and downstream behaviour.","excerpt":"tests whether simulated populations reproduce human variation in reasoning, self-reported traits, their associations, and downstream behaviour.","support":"source-stated"},{"text":"Across seven simulation runs of four models, agents recovered Need for Cognition and Big Five distributions on questionnaires but fell short on cognitive-task performance and downstream behaviours.","excerpt":"Across seven full-battery simulation runs of four models, agents recover Need for Cognition and Big Five trait distributions on self-report questionnaires, but fall short on simulating diversity of cognitive task performance and downstream behaviours.","support":"source-stated"},{"text":"In the four Claude Sonnet 4.6 runs, every agent scored full marks on both CRTs regardless of assigned reflection level.","excerpt":"In the four Claude Sonnet 4.6 runs, every agent scores full marks on both CRTs, whatever reflection level its persona was assigned.","support":"source-stated"},{"text":"Across the BRT scenarios, every run over-produced many coded behaviours, and human links between reflection and behaviour carried over only in part.","excerpt":"In the BRT scenarios, every run over-produces 8 to 11 of 15 coded behaviours by at least 20 percentage points, and people's links between reflection and behaviour carry over only in part.","support":"source-stated"},{"text":"The authors argue that useful synthetic populations must preserve structured variation in reasoning, errors and behaviour, not only trait descriptors.","excerpt":"Useful synthetic populations must preserve not only trait descriptors but the structured variation in reasoning, error type, and behaviour those descriptors are meant to organise.","support":"source-stated"}],"limitations":["The abstract reports results from seven full-battery simulation runs of four models; it does not identify all four models by name.","The abstract describes performance on the stated tests and scenarios; it does not establish how the results generalize to other research settings."],"mediaId":"ce51cf9ee6541848e0f1d4f102a72b379a1fc239d6effe42ef6c62f4d4302800","correctionReason":null,"id":"1bf136d6d812e1f8483beb925af6ec2894a55984b7f5af170197a98cbbb97a4d","publicationId":"fef6eea9-f02f-4206-8fe5-89b14e0c761b","version":1,"sourceUrl":"https://arxiv.org/abs/2611.10508","sourceTitle":"Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-07","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-10T08:39:26.779Z","createdAt":"2026-11-12T11:46:57.286Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"ce51cf9ee6541848e0f1d4f102a72b379a1fc239d6effe42ef6c62f4d4302800","kind":"article","url":"https://arxiv.org/abs/2611.10508","sourceUrl":"https://arxiv.org/abs/2611.10508","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-12T11:34:10.767Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.10508","checkedAt":"2026-11-12T12:50:09.938Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v6","mediaId":"ce51cf9ee6541848e0f1d4f102a72b379a1fc239d6effe42ef6c62f4d4302800","confidence":0.8500000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"c032ba4e160501247c60f006bd7593d5e1a1fb2b1ef50e1727f48ae179bf4b4e","decidedAt":"2026-11-12T11:34:13.197Z"},"provenance":{"textHash":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","observationId":"718d1333fa8d938281c0bfd86e721cffa6e88861801b4d980c488713c301f5c9","retrievedAt":"2026-11-12T11:34:10.431Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"d364baf3-3ed3-4b51-87cd-e5848392495e"},"reviewer":{"model":"gpt-6-luna","runId":"577d1aa6-2f60-4d62-88f2-e82bdd94ba1e"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-12T11:46:57.286Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Weatherhead, Sion","url":null,"kind":"unknown"},{"name":"Newell, Ben","url":null,"kind":"unknown"},{"name":"Belbasis, Aaron","url":null,"kind":"unknown"},{"name":"Salim, Flora","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Weatherhead, Sion","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Weatherhead, Sion","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10508","locator":"head/meta[name=author|citation_author]"},"assertionId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T12:50:09.938Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9","sourceUrl":"https://arxiv.org/abs/2611.10508","contentSha256":"93f6ba196f68dd1ddbf71852cf2940262b820a53a36697b7c645cc153cee4d08","textHash":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","observedAt":"2026-11-12T12:50:09.938Z"}},{"actor":{"name":"Newell, Ben","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Newell, Ben","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10508","locator":"head/meta[name=author|citation_author]"},"assertionId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T12:50:09.938Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9","sourceUrl":"https://arxiv.org/abs/2611.10508","contentSha256":"93f6ba196f68dd1ddbf71852cf2940262b820a53a36697b7c645cc153cee4d08","textHash":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","observedAt":"2026-11-12T12:50:09.938Z"}},{"actor":{"name":"Belbasis, Aaron","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Belbasis, Aaron","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10508","locator":"head/meta[name=author|citation_author]"},"assertionId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T12:50:09.938Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9","sourceUrl":"https://arxiv.org/abs/2611.10508","contentSha256":"93f6ba196f68dd1ddbf71852cf2940262b820a53a36697b7c645cc153cee4d08","textHash":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","observedAt":"2026-11-12T12:50:09.938Z"}},{"actor":{"name":"Salim, Flora","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Salim, Flora","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10508","locator":"head/meta[name=author|citation_author]"},"assertionId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T12:50:09.938Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9","sourceUrl":"https://arxiv.org/abs/2611.10508","contentSha256":"93f6ba196f68dd1ddbf71852cf2940262b820a53a36697b7c645cc153cee4d08","textHash":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","observedAt":"2026-11-12T12:50:09.938Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.10508","snapshotId":"aa9a95c4326c97106e0d9e54b5a7e33df76a6b0946645bbb8c46c4232c5406d9","contentSha256":"93f6ba196f68dd1ddbf71852cf2940262b820a53a36697b7c645cc153cee4d08","textHash":"43ccf80c6622cd9e5aaf66b1fed1d958ddc3014747140c6e8ed8308ec57bd170","observedAt":"2026-11-12T12:50:09.938Z","basis":"publisher-page-metadata","sourceLinks":44,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/1bf136d6d812e1f8483beb925af6ec2894a55984b7f5af170197a98cbbb97a4d/source"}}}],"audits":[{"publicationId":"fef6eea9-f02f-4206-8fe5-89b14e0c761b","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading correctly attributes the paper and its argument to Sion Weatherhead, Ben Newell, Aaron Belbasis and Flora Salim. It frames the conclusion as the authors' point, not as established fact.","excerpt":"Authors:Sion Weatherhead, Ben Newell, Aaron Belbasis, Flora Salim"},{"dimension":"summary","status":"uncertain","explanation":"The summary is faithful and keeps the authors' central thesis. However, it uses 'CRTs' and 'BRT scenarios' without defining them (Cognitive Reflection Test; Behavioural Reflection Test). It also does not explain 'Need for Cognition', 'Big Five' or what the reflection levels mean. A non-specialist cannot tell what full marks or overproduced behaviours signify. The smallest repair is to spell out CRT and BRT and gloss these terms with tooltips. The reading also omits the 'intuitive-lure' explanation and the detail that Claude Sonnet 5 came closest to the human lure pattern, which makes the full-mark finding harder to interpret.","excerpt":"a Cognitive Reflection Test (CRT) and a Behavioural Reflection Test (BRT) whose decision scenarios measure reflection in behaviour"},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-11-07, which matches the v1 submission date. The later feed date of 2026-11-10 is correctly not used.","excerpt":"[Submitted on 7 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"The claims and excerpts match the abstract, including the 473-adult cohort, the seven runs of four models, the Sonnet 4.6 full marks and the BRT overproduction. 'Far exceeded the human sample' is supported by the 49 to 85% versus under 10% figures. The limitations and the generalization caveat are appropriately cautious. The question follows from the authors' stated thesis.","excerpt":"but 49 to 85% score full marks, against under 10% in the human sample"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper. It is the most direct access to the source.","excerpt":null}],"createdAt":"2026-11-12T11:46:57.286Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}