Rozado, David · May 16, 2025
David Rozado reports gender and order effects in LLM hiring evaluations
Markdown · JSON · Original source
David Rozado’s study reports that 22 leading favored female-named candidates in paired CV evaluations across 70 professions, despite identical professional qualifications. The experiment compared profession-matched CVs and swapped names between presentations to test whether gendered names affected selection. Adding an explicit gender field further increased preference for female applicants. With gender-neutral identifiers, several models favored “Candidate A”; counterbalancing gender assignment resulted in gender parity. In isolated CV ratings, female CVs received slightly higher average scores, but Rozado describes the effect size as negligible. Preferred pronouns slightly increased a candidate’s odds of selection regardless of gender, while most models showed a substantial positional bias toward the candidate listed first. These findings concern the evaluations described in the abstract. Rozado argues that they call for caution in deploying LLMs in high-stakes autonomous decision-making and raise doubts about whether LLMs consistently apply principled reasoning.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
· Article
Source details
Original source date: May 16, 2025 (2025-05-16).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-17T13:27:30.689Z. First observed: 2026-11-16T22:56:25.147Z. Retrieved: 2026-11-17T13:26:46.315Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The supplied evidence is the abstract and listing, not the full paper; it does not provide model names or detailed experimental procedures. The reported results concern the described CV evaluations; the supplied evidence does not establish outcomes in actual hiring.
Open question
Would the reported selection patterns persist in hiring evaluations beyond the paired-CV experiment described?
Source-stated
The study tested 22 leading LLMs using profession-matched pairs of CVs, with male and female first names swapped between presentations.
In an experiment involving 22 leading LLMs, each model was systematically given one job description along with a pair of profession-matched CVs
Source-stated
Across 70 professions, the models favored female-named candidates despite identical professional qualifications; adding an explicit gender field increased that preference.
Despite identical professional qualifications across genders, all LLMs consistently favored female-named candidates across 70 different professions. Adding an explicit gender field (male/female) to the CVs further increased the preference for female applicants.
Source-stated
With gender-neutral identifiers, several models preferred “Candidate A”; counterbalancing gender assignment between those identifiers resulted in gender parity.
When gendered names were replaced with gender-neutral identifiers "Candidate A" and "Candidate B", several models displayed a preference to select "Candidate A". Counterbalancing gender assignment between these gender-neutral identifiers resulted in gender parity in candidate selection.
Source-stated
Isolated ratings gave female CVs slightly higher average scores, but the reported effect size was negligible; most models also favored the candidate listed first.
When asked to rate CVs in isolation rather than compare pairs, LLMs assigned slightly higher average scores to female CVs overall, but the effect size was negligible. Including preferred pronouns (he/him or she/her) next to a candidate's name slightly increased the odds of the candidate being
Source-stated
Rozado says the findings warrant caution about high-stakes autonomous decision-making and raise doubts about consistent principled reasoning by LLMs.
These findings underscore the need for caution when deploying LLMs in high-stakes autonomous decision-making contexts and raise doubts about whether LLMs consistently apply principled reasoning.
Source and contributor context
Rozado, David · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: eee75ba5-cdfe-4ada-8181-ea7dcf03e621.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-17T13:27:30.689Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 5be69c1e0c25777a61c3796b49107fd2b93be82c8c8d08f10769a4a56152fa46. Source text hash: df4613211701a7c22a4327d5eebab6be64b92e3a7d4c7f5c7ae969a5c98895a9.
Revision and source-check history
Version 1 · 2026-11-17T13:27:30.689Z
David Rozado reports gender and order effects in LLM hiring evaluations
David Rozado’s study reports that 22 leading favored female-named candidates in paired CV evaluations across 70 professions, despite identical professional qualifications. The experiment compared profession-matched CVs and swapped names between presentations to test whether gendered names affected selection. Adding an explicit gender field further increased preference for female applicants. With gender-neutral identifiers, several models favored “Candidate A”; counterbalancing gender assignment resulted in gender parity. In isolated CV ratings, female CVs received slightly higher average scores, but Rozado describes the effect size as negligible. Preferred pronouns slightly increased a candidate’s odds of selection regardless of gender, while most models showed a substantial positional bias toward the candidate listed first. These findings concern the evaluations described in the abstract. Rozado argues that they call for caution in deploying LLMs in high-stakes autonomous decision-making and raise doubts about whether LLMs consistently apply principled reasoning.
Original source
· Article
Source details
Original source date: May 16, 2025 (2025-05-16).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-17T13:27:30.689Z. First observed: 2026-11-16T22:56:25.147Z. Retrieved: 2026-11-17T13:26:46.315Z.
Limits
The supplied evidence is the abstract and listing, not the full paper; it does not provide model names or detailed experimental procedures. The reported results concern the described CV evaluations; the supplied evidence does not establish outcomes in actual hiring.
Open question
Would the reported selection patterns persist in hiring evaluations beyond the paired-CV experiment described?
Source-stated
The study tested 22 leading LLMs using profession-matched pairs of CVs, with male and female first names swapped between presentations.
In an experiment involving 22 leading LLMs, each model was systematically given one job description along with a pair of profession-matched CVs
Source-stated
Across 70 professions, the models favored female-named candidates despite identical professional qualifications; adding an explicit gender field increased that preference.
Despite identical professional qualifications across genders, all LLMs consistently favored female-named candidates across 70 different professions. Adding an explicit gender field (male/female) to the CVs further increased the preference for female applicants.
Source-stated
With gender-neutral identifiers, several models preferred “Candidate A”; counterbalancing gender assignment between those identifiers resulted in gender parity.
When gendered names were replaced with gender-neutral identifiers "Candidate A" and "Candidate B", several models displayed a preference to select "Candidate A". Counterbalancing gender assignment between these gender-neutral identifiers resulted in gender parity in candidate selection.
Source-stated
Isolated ratings gave female CVs slightly higher average scores, but the reported effect size was negligible; most models also favored the candidate listed first.
When asked to rate CVs in isolation rather than compare pairs, LLMs assigned slightly higher average scores to female CVs overall, but the effect size was negligible. Including preferred pronouns (he/him or she/her) next to a candidate's name slightly increased the odds of the candidate being
Source-stated
Rozado says the findings warrant caution about high-stakes autonomous decision-making and raise doubts about consistent principled reasoning by LLMs.
These findings underscore the need for caution when deploying LLMs in high-stakes autonomous decision-making contexts and raise doubts about whether LLMs consistently apply principled reasoning.
Source and contributor context
Rozado, David · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: eee75ba5-cdfe-4ada-8181-ea7dcf03e621.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-17T13:27:30.689Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 5be69c1e0c25777a61c3796b49107fd2b93be82c8c8d08f10769a4a56152fa46. Source text hash: df4613211701a7c22a4327d5eebab6be64b92e3a7d4c7f5c7ae969a5c98895a9.
uncertain · 2026-11-17T13:27:30.689Z · anthropic/claude-opus-5.5
Publication: eee75ba5-cdfe-4ada-8181-ea7dcf03e621
attribution — pass: The listing names David Rozado as the sole author. The reading attributes the results and the cautionary conclusion to him and his study, and does not present them as established fact or as Escape Meridian's view.
Authors:David Rozado
summary — pass: The title, summary and question match the abstract: 22 LLMs, 70 professions, a preference for female-named candidates, the gender-field and pronoun effects, the Candidate A preference, parity under counterbalancing, negligible isolated-rating effects, and first-position bias. Rozado's caution about principled reasoning is kept as his argument. Positional bias is explained in plain terms. The question raises a real open issue without leading the reader.
most models exhibited a substantial positional bias to select the candidate listed first in the prompt
date — pass: originalPublished is 2025-05-16. That matches the v1 submission date, not the later v2 revision of 27 May 2025.
[Submitted on 16 May 2025 (v1), last revised 27 May 2025 (this version, v2)]
support — uncertain: All claims are stated in the source. However, claim 4's excerpt does not support its second half, that most models favored the first-listed candidate. The excerpt instead runs into the pronoun sentence and stops mid-sentence. Smallest repair: replace the truncated pronoun text in that excerpt with the positional-bias sentence. The limitations correctly note that only the abstract was supplied and that real-world hiring outcomes are not established.
Finally, most models exhibited a substantial positional bias to select the candidate listed first in the prompt.
media — pass: The selected media is the verified direct link to the original arXiv abstract page, which is the source itself.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A language model with many learned numerical settings, used to process and generate language.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA language model with many learned numerical settings, used to process and generate language.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv