{"id":"afc5c6aeda31e3d99f88dd455f7036228fe8d171c4e77aa73ec62ef803d86a5b","title":"arXiv: Auditing the Use of Language Models to Guide Hiring Decisions","argument":"In their paper, Johann D. Gaebler, Sharad Goel, Aziz Huq and Prasanna Tambe propose using correspondence experiments to audit language models used for hiring decisions. These experiments test whether changing application details that suggest an applicant’s race or gender changes an assessment. Applying the method to candidate assessments for K-12 teaching positions in a large public school district, the authors report evidence of moderate race and gender disparities across several state-of-the-art LLMs. They say the pattern was largely robust to changes in the application materials provided and in how the task was framed. The study offers a way to examine demographic differences in model assessments, while its findings concern the tested models and application corpus; the abstract does not establish how the results translate to other hiring settings. The authors also discuss limitations of correspondence experiments, but the abstract does not specify them.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract reports results from several state-of-the-art LLMs and a novel corpus of applications for K-12 teaching positions in one large public school district; it does not establish how the findings apply to other models or hiring settings. The abstract says the authors discuss important limitations of correspondence experiments but does not specify those limitations.","speaker":{"name":"Gaebler, Johann D., Goel, Sharad, Huq, Aziz, Tambe, Prasanna","roleAtSource":"Article author"},"source":{"title":"Auditing the Use of Language Models to Guide Hiring Decisions","publisher":"arXiv","format":"Article","url":"https://arxiv.org/abs/2404.03086","publishedAt":"2024-04-03","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors propose correspondence experiments as an approach to auditing algorithms, including language models used in hiring.","excerpt":"Here we propose and investigate one approach for auditing algorithms: correspondence experiments, a widely applied tool for detecting bias in human judgements.","url":"https://arxiv.org/abs/2404.03086","relationship":"Source-stated"},{"text":"The experiments assess whether race and gender affect decisions by changing application details that suggest an applicant’s demographic traits.","excerpt":"experimentally manipulating elements of submitted application materials that suggest an applicant's demographic traits, such as their listed name.","url":"https://arxiv.org/abs/2404.03086","relationship":"Source-stated"},{"text":"Using applications for K-12 teaching positions in a large public school district, the authors report moderate race and gender disparities in candidate assessments from several state-of-the-art LLMs.","excerpt":"We find evidence of moderate race and gender disparities, a pattern largely robust to varying the types of application material input to the models","url":"https://arxiv.org/abs/2404.03086","relationship":"Source-stated"},{"text":"The authors report that the disparity pattern was largely robust to changes in application materials and task framing.","excerpt":"a pattern largely robust to varying the types of application material input to the models, as well as the framing of the task to the LLMs.","url":"https://arxiv.org/abs/2404.03086","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2404.03086","checkedAt":"2026-11-17T13:58:58.585Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Which limitations of correspondence experiments do the authors identify, and how do those limitations affect interpretation of the reported disparities?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv: Auditing the Use of Language Models to Guide Hiring Decisions","summary":"In their paper, Johann D. Gaebler, Sharad Goel, Aziz Huq and Prasanna Tambe propose using correspondence experiments to audit language models used for hiring decisions. These experiments test whether changing application details that suggest an applicant’s race or gender changes an assessment. Applying the method to candidate assessments for K-12 teaching positions in a large public school district, the authors report evidence of moderate race and gender disparities across several state-of-the-art LLMs. They say the pattern was largely robust to changes in the application materials provided and in how the task was framed. The study offers a way to examine demographic differences in model assessments, while its findings concern the tested models and application corpus; the abstract does not establish how the results translate to other hiring settings. The authors also discuss limitations of correspondence experiments, but the abstract does not specify them.","question":"Which limitations of correspondence experiments do the authors identify, and how do those limitations affect interpretation of the reported disparities?","claims":[{"text":"The authors propose correspondence experiments as an approach to auditing algorithms, including language models used in hiring.","excerpt":"Here we propose and investigate one approach for auditing algorithms: correspondence experiments, a widely applied tool for detecting bias in human judgements.","support":"source-stated"},{"text":"The experiments assess whether race and gender affect decisions by changing application details that suggest an applicant’s demographic traits.","excerpt":"experimentally manipulating elements of submitted application materials that suggest an applicant's demographic traits, such as their listed name.","support":"source-stated"},{"text":"Using applications for K-12 teaching positions in a large public school district, the authors report moderate race and gender disparities in candidate assessments from several state-of-the-art LLMs.","excerpt":"We find evidence of moderate race and gender disparities, a pattern largely robust to varying the types of application material input to the models","support":"source-stated"},{"text":"The authors report that the disparity pattern was largely robust to changes in application materials and task framing.","excerpt":"a pattern largely robust to varying the types of application material input to the models, as well as the framing of the task to the LLMs.","support":"source-stated"}],"limitations":["The abstract reports results from several state-of-the-art LLMs and a novel corpus of applications for K-12 teaching positions in one large public school district; it does not establish how the findings apply to other models or hiring settings.","The abstract says the authors discuss important limitations of correspondence experiments but does not specify those limitations."],"mediaId":"5a6ea56d5a0280deabf57f872795a98ebacb459375098f77f4859e807657222f","correctionReason":null,"id":"afc5c6aeda31e3d99f88dd455f7036228fe8d171c4e77aa73ec62ef803d86a5b","publicationId":"e75b0cdd-b375-4e7f-8690-db437b371f30","version":1,"sourceUrl":"https://arxiv.org/abs/2404.03086","sourceTitle":"Auditing the Use of Language Models to Guide Hiring Decisions","publisher":"arXiv","publishedAt":null,"originalPublished":{"value":"2024-04-03","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-16T22:56:25.147Z","createdAt":"2026-11-17T13:26:44.670Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"5a6ea56d5a0280deabf57f872795a98ebacb459375098f77f4859e807657222f","kind":"article","url":"https://arxiv.org/abs/2404.03086","sourceUrl":"https://arxiv.org/abs/2404.03086","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-17T13:25:45.564Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2404.03086","checkedAt":"2026-11-17T13:58:58.585Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v9","mediaId":"5a6ea56d5a0280deabf57f872795a98ebacb459375098f77f4859e807657222f","confidence":0.96,"provider":"typesafe","model":"jev-1.13.0","inputHash":"246202887f5f1a772002df90bd278b56fd75c4e297ed55acbe751288c6af23bf","decidedAt":"2026-11-17T13:25:46.773Z"},"provenance":{"textHash":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","observationId":"157e59b924f50458b55a71f9d67cb1b47b7f333527f4fd458983cf1eb16b1029","retrievedAt":"2026-11-17T13:25:45.406Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"d586b936-8aaa-4556-85d0-81972456dd49"},"reviewer":{"model":"gpt-6-luna","runId":"5f0eabe4-db47-46db-8bf8-280cc176c879"}},"audit":{"status":"passed","reviewedAt":"2026-11-17T13:26:44.670Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Gaebler, Johann D.","url":null,"kind":"unknown"},{"name":"Goel, Sharad","url":null,"kind":"unknown"},{"name":"Huq, Aziz","url":null,"kind":"unknown"},{"name":"Tambe, Prasanna","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Gaebler, Johann D.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Gaebler, Johann D.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2404.03086","locator":"head/meta[name=author|citation_author]"},"assertionId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T13:58:58.585Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf","sourceUrl":"https://arxiv.org/abs/2404.03086","contentSha256":"ddcd450cc789130929e3ef07b6ee562755f245cc3a85fa679e870525f4289961","textHash":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","observedAt":"2026-11-17T13:58:58.585Z"}},{"actor":{"name":"Goel, Sharad","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Goel, Sharad","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2404.03086","locator":"head/meta[name=author|citation_author]"},"assertionId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T13:58:58.585Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf","sourceUrl":"https://arxiv.org/abs/2404.03086","contentSha256":"ddcd450cc789130929e3ef07b6ee562755f245cc3a85fa679e870525f4289961","textHash":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","observedAt":"2026-11-17T13:58:58.585Z"}},{"actor":{"name":"Huq, Aziz","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Huq, Aziz","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2404.03086","locator":"head/meta[name=author|citation_author]"},"assertionId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T13:58:58.585Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf","sourceUrl":"https://arxiv.org/abs/2404.03086","contentSha256":"ddcd450cc789130929e3ef07b6ee562755f245cc3a85fa679e870525f4289961","textHash":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","observedAt":"2026-11-17T13:58:58.585Z"}},{"actor":{"name":"Tambe, Prasanna","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Tambe, Prasanna","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2404.03086","locator":"head/meta[name=author|citation_author]"},"assertionId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T13:58:58.585Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf","sourceUrl":"https://arxiv.org/abs/2404.03086","contentSha256":"ddcd450cc789130929e3ef07b6ee562755f245cc3a85fa679e870525f4289961","textHash":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","observedAt":"2026-11-17T13:58:58.585Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2404.03086","snapshotId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf","contentSha256":"ddcd450cc789130929e3ef07b6ee562755f245cc3a85fa679e870525f4289961","textHash":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","observedAt":"2026-11-17T13:58:58.585Z","basis":"publisher-page-metadata","sourceLinks":43,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/afc5c6aeda31e3d99f88dd455f7036228fe8d171c4e77aa73ec62ef803d86a5b/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-auditing-the-use-of-language-models-to-guide-hiring-afc5c6ae","formats":{"html":"/readings/arxiv-auditing-the-use-of-language-models-to-guide-hiring-afc5c6ae","markdown":"/readings/arxiv-auditing-the-use-of-language-models-to-guide-hiring-afc5c6ae.md","json":"/readings/arxiv-auditing-the-use-of-language-models-to-guide-hiring-afc5c6ae.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":null,"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"afc5c6aeda31e3d99f88dd455f7036228fe8d171c4e77aa73ec62ef803d86a5b","revision":"e75b0cdd-b375-4e7f-8690-db437b371f30","revisionKind":"publication-id","title":"arXiv: Auditing the Use of Language Models to Guide Hiring Decisions","url":"https://escapemeridian.com/readings/arxiv-auditing-the-use-of-language-models-to-guide-hiring-afc5c6ae"},"source":{"type":"source","id":"https://arxiv.org/abs/2404.03086","revision":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","revisionKind":"sha256-source-text","title":"Auditing the Use of Language Models to Guide Hiring Decisions","url":"https://arxiv.org/abs/2404.03086"},"status":"partial","items":[],"next":null,"scope":"Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter."},"revisions":[{"id":"afc5c6aeda31e3d99f88dd455f7036228fe8d171c4e77aa73ec62ef803d86a5b","title":"arXiv: Auditing the Use of Language Models to Guide Hiring Decisions","argument":"In their paper, Johann D. Gaebler, Sharad Goel, Aziz Huq and Prasanna Tambe propose using correspondence experiments to audit language models used for hiring decisions. These experiments test whether changing application details that suggest an applicant’s race or gender changes an assessment. Applying the method to candidate assessments for K-12 teaching positions in a large public school district, the authors report evidence of moderate race and gender disparities across several state-of-the-art LLMs. They say the pattern was largely robust to changes in the application materials provided and in how the task was framed. The study offers a way to examine demographic differences in model assessments, while its findings concern the tested models and application corpus; the abstract does not establish how the results translate to other hiring settings. The authors also discuss limitations of correspondence experiments, but the abstract does not specify them.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract reports results from several state-of-the-art LLMs and a novel corpus of applications for K-12 teaching positions in one large public school district; it does not establish how the findings apply to other models or hiring settings. The abstract says the authors discuss important limitations of correspondence experiments but does not specify those limitations.","speaker":{"name":"Gaebler, Johann D., Goel, Sharad, Huq, Aziz, Tambe, Prasanna","roleAtSource":"Article author"},"source":{"title":"Auditing the Use of Language Models to Guide Hiring Decisions","publisher":"arXiv","format":"Article","url":"https://arxiv.org/abs/2404.03086","publishedAt":"2024-04-03","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors propose correspondence experiments as an approach to auditing algorithms, including language models used in hiring.","excerpt":"Here we propose and investigate one approach for auditing algorithms: correspondence experiments, a widely applied tool for detecting bias in human judgements.","url":"https://arxiv.org/abs/2404.03086","relationship":"Source-stated"},{"text":"The experiments assess whether race and gender affect decisions by changing application details that suggest an applicant’s demographic traits.","excerpt":"experimentally manipulating elements of submitted application materials that suggest an applicant's demographic traits, such as their listed name.","url":"https://arxiv.org/abs/2404.03086","relationship":"Source-stated"},{"text":"Using applications for K-12 teaching positions in a large public school district, the authors report moderate race and gender disparities in candidate assessments from several state-of-the-art LLMs.","excerpt":"We find evidence of moderate race and gender disparities, a pattern largely robust to varying the types of application material input to the models","url":"https://arxiv.org/abs/2404.03086","relationship":"Source-stated"},{"text":"The authors report that the disparity pattern was largely robust to changes in application materials and task framing.","excerpt":"a pattern largely robust to varying the types of application material input to the models, as well as the framing of the task to the LLMs.","url":"https://arxiv.org/abs/2404.03086","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2404.03086","checkedAt":"2026-11-17T13:58:58.585Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Which limitations of correspondence experiments do the authors identify, and how do those limitations affect interpretation of the reported disparities?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv: Auditing the Use of Language Models to Guide Hiring Decisions","summary":"In their paper, Johann D. Gaebler, Sharad Goel, Aziz Huq and Prasanna Tambe propose using correspondence experiments to audit language models used for hiring decisions. These experiments test whether changing application details that suggest an applicant’s race or gender changes an assessment. Applying the method to candidate assessments for K-12 teaching positions in a large public school district, the authors report evidence of moderate race and gender disparities across several state-of-the-art LLMs. They say the pattern was largely robust to changes in the application materials provided and in how the task was framed. The study offers a way to examine demographic differences in model assessments, while its findings concern the tested models and application corpus; the abstract does not establish how the results translate to other hiring settings. The authors also discuss limitations of correspondence experiments, but the abstract does not specify them.","question":"Which limitations of correspondence experiments do the authors identify, and how do those limitations affect interpretation of the reported disparities?","claims":[{"text":"The authors propose correspondence experiments as an approach to auditing algorithms, including language models used in hiring.","excerpt":"Here we propose and investigate one approach for auditing algorithms: correspondence experiments, a widely applied tool for detecting bias in human judgements.","support":"source-stated"},{"text":"The experiments assess whether race and gender affect decisions by changing application details that suggest an applicant’s demographic traits.","excerpt":"experimentally manipulating elements of submitted application materials that suggest an applicant's demographic traits, such as their listed name.","support":"source-stated"},{"text":"Using applications for K-12 teaching positions in a large public school district, the authors report moderate race and gender disparities in candidate assessments from several state-of-the-art LLMs.","excerpt":"We find evidence of moderate race and gender disparities, a pattern largely robust to varying the types of application material input to the models","support":"source-stated"},{"text":"The authors report that the disparity pattern was largely robust to changes in application materials and task framing.","excerpt":"a pattern largely robust to varying the types of application material input to the models, as well as the framing of the task to the LLMs.","support":"source-stated"}],"limitations":["The abstract reports results from several state-of-the-art LLMs and a novel corpus of applications for K-12 teaching positions in one large public school district; it does not establish how the findings apply to other models or hiring settings.","The abstract says the authors discuss important limitations of correspondence experiments but does not specify those limitations."],"mediaId":"5a6ea56d5a0280deabf57f872795a98ebacb459375098f77f4859e807657222f","correctionReason":null,"id":"afc5c6aeda31e3d99f88dd455f7036228fe8d171c4e77aa73ec62ef803d86a5b","publicationId":"e75b0cdd-b375-4e7f-8690-db437b371f30","version":1,"sourceUrl":"https://arxiv.org/abs/2404.03086","sourceTitle":"Auditing the Use of Language Models to Guide Hiring Decisions","publisher":"arXiv","publishedAt":null,"originalPublished":{"value":"2024-04-03","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-16T22:56:25.147Z","createdAt":"2026-11-17T13:26:44.670Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"5a6ea56d5a0280deabf57f872795a98ebacb459375098f77f4859e807657222f","kind":"article","url":"https://arxiv.org/abs/2404.03086","sourceUrl":"https://arxiv.org/abs/2404.03086","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-17T13:25:45.564Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2404.03086","checkedAt":"2026-11-17T13:58:58.585Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v9","mediaId":"5a6ea56d5a0280deabf57f872795a98ebacb459375098f77f4859e807657222f","confidence":0.96,"provider":"typesafe","model":"jev-1.13.0","inputHash":"246202887f5f1a772002df90bd278b56fd75c4e297ed55acbe751288c6af23bf","decidedAt":"2026-11-17T13:25:46.773Z"},"provenance":{"textHash":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","observationId":"157e59b924f50458b55a71f9d67cb1b47b7f333527f4fd458983cf1eb16b1029","retrievedAt":"2026-11-17T13:25:45.406Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"d586b936-8aaa-4556-85d0-81972456dd49"},"reviewer":{"model":"gpt-6-luna","runId":"5f0eabe4-db47-46db-8bf8-280cc176c879"}},"audit":{"status":"passed","reviewedAt":"2026-11-17T13:26:44.670Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Gaebler, Johann D.","url":null,"kind":"unknown"},{"name":"Goel, Sharad","url":null,"kind":"unknown"},{"name":"Huq, Aziz","url":null,"kind":"unknown"},{"name":"Tambe, Prasanna","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Gaebler, Johann D.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Gaebler, Johann D.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2404.03086","locator":"head/meta[name=author|citation_author]"},"assertionId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T13:58:58.585Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf","sourceUrl":"https://arxiv.org/abs/2404.03086","contentSha256":"ddcd450cc789130929e3ef07b6ee562755f245cc3a85fa679e870525f4289961","textHash":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","observedAt":"2026-11-17T13:58:58.585Z"}},{"actor":{"name":"Goel, Sharad","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Goel, Sharad","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2404.03086","locator":"head/meta[name=author|citation_author]"},"assertionId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T13:58:58.585Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf","sourceUrl":"https://arxiv.org/abs/2404.03086","contentSha256":"ddcd450cc789130929e3ef07b6ee562755f245cc3a85fa679e870525f4289961","textHash":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","observedAt":"2026-11-17T13:58:58.585Z"}},{"actor":{"name":"Huq, Aziz","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Huq, Aziz","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2404.03086","locator":"head/meta[name=author|citation_author]"},"assertionId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T13:58:58.585Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf","sourceUrl":"https://arxiv.org/abs/2404.03086","contentSha256":"ddcd450cc789130929e3ef07b6ee562755f245cc3a85fa679e870525f4289961","textHash":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","observedAt":"2026-11-17T13:58:58.585Z"}},{"actor":{"name":"Tambe, Prasanna","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Tambe, Prasanna","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2404.03086","locator":"head/meta[name=author|citation_author]"},"assertionId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T13:58:58.585Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf","sourceUrl":"https://arxiv.org/abs/2404.03086","contentSha256":"ddcd450cc789130929e3ef07b6ee562755f245cc3a85fa679e870525f4289961","textHash":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","observedAt":"2026-11-17T13:58:58.585Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2404.03086","snapshotId":"9d6d57e1dfa42e4ff01e94c8f9e594d79e923b404a81bb557c360e0a37efc0cf","contentSha256":"ddcd450cc789130929e3ef07b6ee562755f245cc3a85fa679e870525f4289961","textHash":"84354ed24d9d584bc37c4bb2a47f1391931b30586fec472d62cccabd5aac5e71","observedAt":"2026-11-17T13:58:58.585Z","basis":"publisher-page-metadata","sourceLinks":43,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/afc5c6aeda31e3d99f88dd455f7036228fe8d171c4e77aa73ec62ef803d86a5b/source"}}}],"audits":[{"publicationId":"e75b0cdd-b375-4e7f-8690-db437b371f30","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading correctly attributes the paper to Johann D. Gaebler, Sharad Goel, Aziz Huq and Prasanna Tambe. It frames the method proposal and findings as the authors' own reports, not as established facts.","excerpt":"Authors:Johann D. Gaebler, Sharad Goel, Aziz Huq, Prasanna Tambe"},{"dimension":"summary","status":"pass","explanation":"The title, summary and question match the abstract. Correspondence experiments are explained in plain language. The question asks about a real gap: the limitations are unspecified. A minor optional repair: say that the abstract presents this as an answer to regulations that call for algorithmic auditing but give little guidance on how to do it. That would keep the authors' motivating stake. 'LLMs' could also be spelled out on first use. Neither omission prevents understanding.","excerpt":"current regulations -- as well as the scientific literature -- provide little guidance on how to conduct these assessments"},{"dimension":"date","status":"pass","explanation":"originalPublished is 2024-04-03, which matches the v1 submission date on arXiv.","excerpt":"[Submitted on 3 Apr 2024]"},{"dimension":"support","status":"pass","explanation":"Each claim matches the abstract. This covers the proposed method, the manipulation of demographic cues such as names, the moderate disparities found on the K-12 corpus, and the robustness to changes in materials and framing. The limitations are stated conditionally and are not overstated. No benefits or harms are invented.","excerpt":"We find evidence of moderate race and gender disparities, a pattern largely robust to varying the types of application material input to the models, as well as the framing of the task to the LLMs."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified arXiv abstract page, which is the direct original source.","excerpt":null}],"createdAt":"2026-11-17T13:26:44.670Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}