Di Cursi, Francesco, Boldrini, Chiara, Passarella, Andrea, Conti, Marco · Nov 12, 2026
arXiv cs.AI study: LLMs do not consistently reproduce individuals’ questionnaire responses
Markdown · JSON · Original source
In an cs.AI paper, Francesco Di Cursi, Chiara Boldrini, Andrea Passarella and Marco Conti test whether language models can answer a personality questionnaire as a specific person would, using that person’s social-media posts. They compare predicted trait scores with participants’ actual scores, and with predictions made without personal text or from general population tendencies. The authors report that personal writing does not consistently improve predictions. Models perform better for people near the middle of the trait distribution; at either end, they often stay close to central values or perform unevenly. More text does not systematically reduce error: open-source models fluctuate more, while GPT-4 is comparatively stable. Given an explicit personality profile without user text, GPT-4 most clearly produces consistent questionnaire responses, particularly for Extraversion and Neuroticism. The authors conclude that the evaluated models do not consistently reproduce a particular person’s questionnaire behavior. The abstract does not establish how well the findings generalize beyond the tested models and dataset.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 12, 2026 (2026-11-12).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-13T17:17:18.186Z. First observed: 2026-11-13T07:43:44.251Z. Retrieved: 2026-11-13T17:04:50.866Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The abstract reports evaluation on GPT-4, Gemma, Llama 3.1, and Mistral using the MyPersonality dataset; it does not establish how well the findings generalize beyond these models and dataset. The abstract does not provide numerical error measures or details of the questionnaire results by trait beyond the patterns it describes.
Open question
Would evaluation on people or settings beyond the MyPersonality dataset change the reported pattern?
Source-stated
The study tests whether language models can answer a personality questionnaire as a specific individual would, using that person’s social-media posts.
This paper investigates whether Large Language Models can answer a personality questionnaire as a specific individual would. Given a user's social-media posts, the model completes the BFI-2-60 Big Five questionnaire as that user.
Source-stated
The authors report that user text does not consistently improve model performance.
The results show that user text does not consistently improve performance.
Source-stated
Models perform better near the middle of the trait distribution; at the lower and upper ends, they may stay near central values or perform unevenly across ends.
Models perform better near the middle of the trait distribution, while at the lower and upper ends they often remain close to central values or perform well at one end but poorly at the other.
Source-stated
More context does not systematically reduce error; open-source models fluctuate more, while GPT-4 remains comparatively stable.
Increasing context does not systematically reduce error: open-source models fluctuate more, whereas GPT-4 remains comparatively stable.
Source-stated
With an explicit personality profile but no user text, GPT-4 most clearly produces consistent questionnaire responses, particularly for Extraversion and Neuroticism.
When given an explicit profile, GPT-4 most clearly translates trait information into consistent questionnaire responses, particularly for Extraversion and Neuroticism.
Source-stated
The authors conclude that the evaluated models do not consistently reproduce specific individuals’ questionnaire behavior.
Overall, the evaluated models do not consistently reproduce the questionnaire behavior of specific individuals.
Source and contributor context
Di Cursi, Francesco, Boldrini, Chiara, Passarella, Andrea, Conti, Marco · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 0545a80a-bc3e-48f1-8150-38504984d89e.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T17:17:18.186Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: e1afe88d11befcedb0a5a5d6f4b3c85e96930870cb34704c413d9c54a5101714. Source text hash: 5ca9d78b9d92f63e7189099eb04db1b7b3e3cf6641da9d586570d3179941847c.
Revision and source-check history
Version 1 · 2026-11-13T17:17:18.186Z
arXiv cs.AI study: LLMs do not consistently reproduce individuals’ questionnaire responses
In an cs.AI paper, Francesco Di Cursi, Chiara Boldrini, Andrea Passarella and Marco Conti test whether language models can answer a personality questionnaire as a specific person would, using that person’s social-media posts. They compare predicted trait scores with participants’ actual scores, and with predictions made without personal text or from general population tendencies. The authors report that personal writing does not consistently improve predictions. Models perform better for people near the middle of the trait distribution; at either end, they often stay close to central values or perform unevenly. More text does not systematically reduce error: open-source models fluctuate more, while GPT-4 is comparatively stable. Given an explicit personality profile without user text, GPT-4 most clearly produces consistent questionnaire responses, particularly for Extraversion and Neuroticism. The authors conclude that the evaluated models do not consistently reproduce a particular person’s questionnaire behavior. The abstract does not establish how well the findings generalize beyond the tested models and dataset.
Original source
cs.AI · Article
Source details
Original source date: Nov 12, 2026 (2026-11-12).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-13T17:17:18.186Z. First observed: 2026-11-13T07:43:44.251Z. Retrieved: 2026-11-13T17:04:50.866Z.
Limits
The abstract reports evaluation on GPT-4, Gemma, Llama 3.1, and Mistral using the MyPersonality dataset; it does not establish how well the findings generalize beyond these models and dataset. The abstract does not provide numerical error measures or details of the questionnaire results by trait beyond the patterns it describes.
Open question
Would evaluation on people or settings beyond the MyPersonality dataset change the reported pattern?
Source-stated
The study tests whether language models can answer a personality questionnaire as a specific individual would, using that person’s social-media posts.
This paper investigates whether Large Language Models can answer a personality questionnaire as a specific individual would. Given a user's social-media posts, the model completes the BFI-2-60 Big Five questionnaire as that user.
Source-stated
The authors report that user text does not consistently improve model performance.
The results show that user text does not consistently improve performance.
Source-stated
Models perform better near the middle of the trait distribution; at the lower and upper ends, they may stay near central values or perform unevenly across ends.
Models perform better near the middle of the trait distribution, while at the lower and upper ends they often remain close to central values or perform well at one end but poorly at the other.
Source-stated
More context does not systematically reduce error; open-source models fluctuate more, while GPT-4 remains comparatively stable.
Increasing context does not systematically reduce error: open-source models fluctuate more, whereas GPT-4 remains comparatively stable.
Source-stated
With an explicit personality profile but no user text, GPT-4 most clearly produces consistent questionnaire responses, particularly for Extraversion and Neuroticism.
When given an explicit profile, GPT-4 most clearly translates trait information into consistent questionnaire responses, particularly for Extraversion and Neuroticism.
Source-stated
The authors conclude that the evaluated models do not consistently reproduce specific individuals’ questionnaire behavior.
Overall, the evaluated models do not consistently reproduce the questionnaire behavior of specific individuals.
Source and contributor context
Di Cursi, Francesco, Boldrini, Chiara, Passarella, Andrea, Conti, Marco · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 0545a80a-bc3e-48f1-8150-38504984d89e.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T17:17:18.186Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: e1afe88d11befcedb0a5a5d6f4b3c85e96930870cb34704c413d9c54a5101714. Source text hash: 5ca9d78b9d92f63e7189099eb04db1b7b3e3cf6641da9d586570d3179941847c.
uncertain · 2026-11-13T17:17:18.186Z · anthropic/claude-opus-5.5
Publication: 0545a80a-bc3e-48f1-8150-38504984d89e
attribution — pass: The reading correctly attributes the paper to Francesco Di Cursi, Chiara Boldrini, Andrea Passarella and Marco Conti on arXiv cs.AI. Findings are framed as the authors' reports and conclusions rather than established fact.
Authors:Francesco Di Cursi, Chiara Boldrini, Andrea Passarella, Marco Conti
summary — uncertain: The findings are faithful, but the reading omits the paper's central framing: LLMs as 'digital twins', meaning AI stand-ins for specific people. That framing is why the test matters, so a general reader gets results without clear significance. It names only GPT-4, not Gemma, Llama 3.1 and Mistral, so 'open-source models' is vague. The question mentions the 'MyPersonality dataset' without explaining it. The Big Five questionnaire is also not named in the body. Smallest repair: add one sentence that the authors frame this as testing whether LLMs can act as digital twins of individuals, and name all four models.
Title:Large Language Models as Digital Twins: a Distribution-Aware Questionnaire-Based Evaluation
date — pass: originalPublished is 2026-11-12, which matches the v1 submission date. The feed date of 2026-11-13 is later and is correctly not used.
[v1] Thu, 12 Nov 2026 10:18:16 UTC (761 KB)
support — pass: Each claim matches the abstract closely. The limitations correctly note that the abstract tests four models on a single dataset and gives no numerical error measures. The question about generalization follows from the evidence and is not a leading claim. No benefits or costs are invented.
Overall, the evaluated models do not consistently reproduce the questionnaire behavior of specific individuals.
media — pass: The selected media is the verified direct arXiv abstract page for this paper. It is the most direct way to access the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv