Escape Meridian

Rozado, David · May 16, 2025

David Rozado reports gender and order effects in LLM hiring evaluations

Markdown · JSON · Original source

David Rozado’s study reports that 22 leading favored female-named candidates in paired CV evaluations across 70 professions, despite identical professional qualifications. The experiment compared profession-matched CVs and swapped names between presentations to test whether gendered names affected selection. Adding an explicit gender field further increased preference for female applicants. With gender-neutral identifiers, several models favored “Candidate A”; counterbalancing gender assignment resulted in gender parity. In isolated CV ratings, female CVs received slightly higher average scores, but Rozado describes the effect size as negligible. Preferred pronouns slightly increased a candidate’s odds of selection regardless of gender, while most models showed a substantial positional bias toward the candidate listed first. These findings concern the evaluations described in the abstract. Rozado argues that they call for caution in deploying LLMs in high-stakes autonomous decision-making and raise doubts about whether LLMs consistently apply principled reasoning.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

arXivArticle · arxiv.orgOpen original source

Original source

· Article

Source details

Original source date: May 16, 2025 (2025-05-16).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-17T13:27:30.689Z. First observed: 2026-11-16T22:56:25.147Z. Retrieved: 2026-11-17T13:26:46.315Z.

Referenced by

Recorded citations in public readings. A reference alone does not establish agreement or influence.

No public references recorded here yet.

Reference data (JSON)

Limits

The supplied evidence is the abstract and listing, not the full paper; it does not provide model names or detailed experimental procedures. The reported results concern the described CV evaluations; the supplied evidence does not establish outcomes in actual hiring.

Open question

Would the reported selection patterns persist in hiring evaluations beyond the paired-CV experiment described?

Source-stated

The study tested 22 leading LLMs using profession-matched pairs of CVs, with male and female first names swapped between presentations.

In an experiment involving 22 leading LLMs, each model was systematically given one job description along with a pair of profession-matched CVs

Source-stated

Across 70 professions, the models favored female-named candidates despite identical professional qualifications; adding an explicit gender field increased that preference.

Despite identical professional qualifications across genders, all LLMs consistently favored female-named candidates across 70 different professions. Adding an explicit gender field (male/female) to the CVs further increased the preference for female applicants.

Source-stated

With gender-neutral identifiers, several models preferred “Candidate A”; counterbalancing gender assignment between those identifiers resulted in gender parity.

When gendered names were replaced with gender-neutral identifiers "Candidate A" and "Candidate B", several models displayed a preference to select "Candidate A". Counterbalancing gender assignment between these gender-neutral identifiers resulted in gender parity in candidate selection.

Source-stated

Isolated ratings gave female CVs slightly higher average scores, but the reported effect size was negligible; most models also favored the candidate listed first.

When asked to rate CVs in isolation rather than compare pairs, LLMs assigned slightly higher average scores to female CVs overall, but the effect size was negligible. Including preferred pronouns (he/him or she/her) next to a candidate's name slightly increased the odds of the candidate being

Source-stated

Rozado says the findings warrant caution about high-stakes autonomous decision-making and raise doubts about consistent principled reasoning by LLMs.

These findings underscore the need for caution when deploying LLMs in high-stakes autonomous decision-making contexts and raise doubts about whether LLMs consistently apply principled reasoning.

Source and contributor context

Rozado, David · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

No suitable media was found by the supported page extraction.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Large language model

A language model with many learned numerical settings, used to process and generate language.

arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

Editorial provenance

Version 1. Publication ID: eee75ba5-cdfe-4ada-8181-ea7dcf03e621.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-17T13:27:30.689Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material_development.

Observation: 5be69c1e0c25777a61c3796b49107fd2b93be82c8c8d08f10769a4a56152fa46. Source text hash: df4613211701a7c22a4327d5eebab6be64b92e3a7d4c7f5c7ae969a5c98895a9.

Revision and source-check history

Version 1 · 2026-11-17T13:27:30.689Z

David Rozado reports gender and order effects in LLM hiring evaluations

David Rozado’s study reports that 22 leading favored female-named candidates in paired CV evaluations across 70 professions, despite identical professional qualifications. The experiment compared profession-matched CVs and swapped names between presentations to test whether gendered names affected selection. Adding an explicit gender field further increased preference for female applicants. With gender-neutral identifiers, several models favored “Candidate A”; counterbalancing gender assignment resulted in gender parity. In isolated CV ratings, female CVs received slightly higher average scores, but Rozado describes the effect size as negligible. Preferred pronouns slightly increased a candidate’s odds of selection regardless of gender, while most models showed a substantial positional bias toward the candidate listed first. These findings concern the evaluations described in the abstract. Rozado argues that they call for caution in deploying LLMs in high-stakes autonomous decision-making and raise doubts about whether LLMs consistently apply principled reasoning.

Original source

· Article

Source details

Original source date: May 16, 2025 (2025-05-16).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-17T13:27:30.689Z. First observed: 2026-11-16T22:56:25.147Z. Retrieved: 2026-11-17T13:26:46.315Z.

Limits

The supplied evidence is the abstract and listing, not the full paper; it does not provide model names or detailed experimental procedures. The reported results concern the described CV evaluations; the supplied evidence does not establish outcomes in actual hiring.

Open question

Would the reported selection patterns persist in hiring evaluations beyond the paired-CV experiment described?

Source-stated

The study tested 22 leading LLMs using profession-matched pairs of CVs, with male and female first names swapped between presentations.

In an experiment involving 22 leading LLMs, each model was systematically given one job description along with a pair of profession-matched CVs

Source-stated

Across 70 professions, the models favored female-named candidates despite identical professional qualifications; adding an explicit gender field increased that preference.

Despite identical professional qualifications across genders, all LLMs consistently favored female-named candidates across 70 different professions. Adding an explicit gender field (male/female) to the CVs further increased the preference for female applicants.

Source-stated

With gender-neutral identifiers, several models preferred “Candidate A”; counterbalancing gender assignment between those identifiers resulted in gender parity.

When gendered names were replaced with gender-neutral identifiers "Candidate A" and "Candidate B", several models displayed a preference to select "Candidate A". Counterbalancing gender assignment between these gender-neutral identifiers resulted in gender parity in candidate selection.

Source-stated

Isolated ratings gave female CVs slightly higher average scores, but the reported effect size was negligible; most models also favored the candidate listed first.

When asked to rate CVs in isolation rather than compare pairs, LLMs assigned slightly higher average scores to female CVs overall, but the effect size was negligible. Including preferred pronouns (he/him or she/her) next to a candidate's name slightly increased the odds of the candidate being

Source-stated

Rozado says the findings warrant caution about high-stakes autonomous decision-making and raise doubts about consistent principled reasoning by LLMs.

These findings underscore the need for caution when deploying LLMs in high-stakes autonomous decision-making contexts and raise doubts about whether LLMs consistently apply principled reasoning.

Source and contributor context

Rozado, David · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

No suitable media was found by the supported page extraction.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Large language model

A language model with many learned numerical settings, used to process and generate language.

arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

Editorial provenance

Version 1. Publication ID: eee75ba5-cdfe-4ada-8181-ea7dcf03e621.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-17T13:27:30.689Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material_development.

Observation: 5be69c1e0c25777a61c3796b49107fd2b93be82c8c8d08f10769a4a56152fa46. Source text hash: df4613211701a7c22a4327d5eebab6be64b92e3a7d4c7f5c7ae969a5c98895a9.

uncertain · 2026-11-17T13:27:30.689Z · anthropic/claude-opus-5.5

Publication: eee75ba5-cdfe-4ada-8181-ea7dcf03e621

attribution — pass: The listing names David Rozado as the sole author. The reading attributes the results and the cautionary conclusion to him and his study, and does not present them as established fact or as Escape Meridian's view.

Authors:David Rozado

summary — pass: The title, summary and question match the abstract: 22 LLMs, 70 professions, a preference for female-named candidates, the gender-field and pronoun effects, the Candidate A preference, parity under counterbalancing, negligible isolated-rating effects, and first-position bias. Rozado's caution about principled reasoning is kept as his argument. Positional bias is explained in plain terms. The question raises a real open issue without leading the reader.

most models exhibited a substantial positional bias to select the candidate listed first in the prompt

date — pass: originalPublished is 2025-05-16. That matches the v1 submission date, not the later v2 revision of 27 May 2025.

[Submitted on 16 May 2025 (v1), last revised 27 May 2025 (this version, v2)]

support — uncertain: All claims are stated in the source. However, claim 4's excerpt does not support its second half, that most models favored the first-listed candidate. The excerpt instead runs into the pronoun sentence and stops mid-sentence. Smallest repair: replace the truncated pronoun text in that excerpt with the positional-bias sentence. The limitations correctly note that only the abstract was supplied and that real-world hiring outcomes are not established.

Finally, most models exhibited a substantial positional bias to select the candidate listed first in the prompt.

media — pass: The selected media is the verified direct link to the original arXiv abstract page, which is the source itself.

Challenge or correct this reading

Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.