{"id":"b4157dc37d84a4ed3fe269c947043b87229c4f3487e2dbd1953369e28eefad63","title":"arXiv cs.AI: Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge","argument":"The authors of “Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge” argue that an Agent-as-a-Judge can gather decisive evidence and still be swayed by a false note attributed to an authority. Across four judges and five datasets, such a note overturned up to 40% of tool-using judges’ correct verdicts, even when the judges collected decisive evidence. The authors treat the note as a controlled diagnostic of confirmation bias, not an adversarial attack. They identify three ways the shift can happen: over-trust, when a judge verifies evidence and then defers; hallucination, when it fabricates support for changing its verdict; and under-investigation, when it stops at the first corroborating signal. For the two Claude judges, the authors say over-trust accounted for most harmful flips; smaller judges showed all three failure modes. Lightweight mitigations reduced the bias but did not remove it. The result raises a practical concern for people relying on these judges to evaluate open-ended agent tasks: collecting evidence alone may not prevent an irrelevant authority cue from changing a verdict. The abstract does not specify which cue properties drive the shift or how the mitigations were tested.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The source is marked “Preprint. Under Review”; the evidence provided is the abstract, not the full paper. The abstract says the shift is driven by specific properties of the cue but does not identify those properties or describe how the lightweight mitigations were evaluated. The amount by which they reduce bias is not stated.","speaker":{"name":"Brown, Drake Benjamin, Pahwa, Khushbu, Ge, Yubin, Romeo, Salvatore, Agnihotri, Akhil, Zhang, Yi, Ma, Xiaofei, Bonadiman, Daniele","roleAtSource":"Article author"},"source":{"title":"Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.05662","publishedAt":"2026-11-03","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors report that a false pre-screening note attributed to an authority overturned up to 40% of tool-using judges’ correct verdicts across four judges and five datasets, despite decisive evidence collection.","excerpt":"a single false pre-screening note, attributed to an authority and carrying no evidential bearing on the correct verdict, overturns up to 40% of a tool-using judge's correct verdicts across four judges and five datasets, even though the judge collects decisive evidence.","url":"https://arxiv.org/abs/2611.05662","relationship":"Source-stated"},{"text":"The authors identify over-trust, hallucination, and under-investigation as three failure modes through which confirmation bias changes judge behavior.","excerpt":"We classify changes in judge behavior and attribute changes to one of three failure modes: over-trust (verify, conclude correctly, then defer), hallucination (fabricate support for the flip), and under-investigation (stop at the first corroborating signal).","url":"https://arxiv.org/abs/2611.05662","relationship":"Source-stated"},{"text":"The authors say over-trust predominated for the two Claude judges, while smaller judges showed all three error modes; lightweight mitigations blunted but did not remove the bias.","excerpt":"For the two Claude judges, the failure is predominantly over-trust (67-75% of harmful flips), while smaller judges have error modes spread across all three. Lightweight mitigations blunt but never remove the bias.","url":"https://arxiv.org/abs/2611.05662","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05662","checkedAt":"2026-11-09T10:54:09.921Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Which properties of the false note drive the change in verdict, and how much do the lightweight mitigations reduce it?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge","summary":"The authors of “Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge” argue that an Agent-as-a-Judge can gather decisive evidence and still be swayed by a false note attributed to an authority. Across four judges and five datasets, such a note overturned up to 40% of tool-using judges’ correct verdicts, even when the judges collected decisive evidence. The authors treat the note as a controlled diagnostic of confirmation bias, not an adversarial attack. They identify three ways the shift can happen: over-trust, when a judge verifies evidence and then defers; hallucination, when it fabricates support for changing its verdict; and under-investigation, when it stops at the first corroborating signal. For the two Claude judges, the authors say over-trust accounted for most harmful flips; smaller judges showed all three failure modes. Lightweight mitigations reduced the bias but did not remove it. The result raises a practical concern for people relying on these judges to evaluate open-ended agent tasks: collecting evidence alone may not prevent an irrelevant authority cue from changing a verdict. The abstract does not specify which cue properties drive the shift or how the mitigations were tested.","question":"Which properties of the false note drive the change in verdict, and how much do the lightweight mitigations reduce it?","claims":[{"text":"The authors report that a false pre-screening note attributed to an authority overturned up to 40% of tool-using judges’ correct verdicts across four judges and five datasets, despite decisive evidence collection.","excerpt":"a single false pre-screening note, attributed to an authority and carrying no evidential bearing on the correct verdict, overturns up to 40% of a tool-using judge's correct verdicts across four judges and five datasets, even though the judge collects decisive evidence.","support":"source-stated"},{"text":"The authors identify over-trust, hallucination, and under-investigation as three failure modes through which confirmation bias changes judge behavior.","excerpt":"We classify changes in judge behavior and attribute changes to one of three failure modes: over-trust (verify, conclude correctly, then defer), hallucination (fabricate support for the flip), and under-investigation (stop at the first corroborating signal).","support":"source-stated"},{"text":"The authors say over-trust predominated for the two Claude judges, while smaller judges showed all three error modes; lightweight mitigations blunted but did not remove the bias.","excerpt":"For the two Claude judges, the failure is predominantly over-trust (67-75% of harmful flips), while smaller judges have error modes spread across all three. Lightweight mitigations blunt but never remove the bias.","support":"source-stated"}],"limitations":["The source is marked “Preprint. Under Review”; the evidence provided is the abstract, not the full paper.","The abstract says the shift is driven by specific properties of the cue but does not identify those properties or describe how the lightweight mitigations were evaluated. The amount by which they reduce bias is not stated."],"mediaId":"9fdb6f74ae483d9b2394371c90ab1b2a2328540f77d3c7acd7e55bd618f0c82e","correctionReason":null,"id":"b4157dc37d84a4ed3fe269c947043b87229c4f3487e2dbd1953369e28eefad63","publicationId":"dc1f9f97-2449-4f70-8c4f-c57477026861","version":1,"sourceUrl":"https://arxiv.org/abs/2611.05662","sourceTitle":"Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-03","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:06:20.581Z","createdAt":"2026-11-09T10:23:16.555Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"9fdb6f74ae483d9b2394371c90ab1b2a2328540f77d3c7acd7e55bd618f0c82e","kind":"article","url":"https://arxiv.org/abs/2611.05662","sourceUrl":"https://arxiv.org/abs/2611.05662","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T10:22:30.261Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05662","checkedAt":"2026-11-09T10:54:09.921Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"9fdb6f74ae483d9b2394371c90ab1b2a2328540f77d3c7acd7e55bd618f0c82e","confidence":0.84,"provider":"typesafe","model":"jev-1.13.0","inputHash":"84d208ce3aa455428c6faa3e2ba02aae641e2ab01dfe529584316dd3fe8032db","decidedAt":"2026-11-09T10:22:31.385Z"},"provenance":{"textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observationId":"5aef6733a2c742849e31c62e79a6e98fceae5cf8c511bf09e5ec577f2940c2fd","retrievedAt":"2026-11-09T10:22:30.064Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"87f3701f-683e-432e-8ff0-319df89d7166"},"reviewer":{"model":"gpt-6-luna","runId":"e56894bb-ddc4-4499-8f67-2c9da34ded7d"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T10:23:16.555Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Brown, Drake Benjamin","url":null,"kind":"unknown"},{"name":"Pahwa, Khushbu","url":null,"kind":"unknown"},{"name":"Ge, Yubin","url":null,"kind":"unknown"},{"name":"Romeo, Salvatore","url":null,"kind":"unknown"},{"name":"Agnihotri, Akhil","url":null,"kind":"unknown"},{"name":"Zhang, Yi","url":null,"kind":"unknown"},{"name":"Ma, Xiaofei","url":null,"kind":"unknown"},{"name":"Bonadiman, Daniele","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Brown, Drake Benjamin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Brown, Drake Benjamin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Pahwa, Khushbu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Pahwa, Khushbu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Ge, Yubin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ge, Yubin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Romeo, Salvatore","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Romeo, Salvatore","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Agnihotri, Akhil","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Agnihotri, Akhil","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Zhang, Yi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhang, Yi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Ma, Xiaofei","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ma, Xiaofei","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Bonadiman, Daniele","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Bonadiman, Daniele","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}}],"participationCoverage":{"total":8,"returned":8,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.05662","snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z","basis":"publisher-page-metadata","sourceLinks":47,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/b4157dc37d84a4ed3fe269c947043b87229c4f3487e2dbd1953369e28eefad63/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-grounded-but-not-immune-confirmation-bias-in-agent-b4157dc3","formats":{"html":"/readings/arxiv-cs-ai-grounded-but-not-immune-confirmation-bias-in-agent-b4157dc3","markdown":"/readings/arxiv-cs-ai-grounded-but-not-immune-confirmation-bias-in-agent-b4157dc3.md","json":"/readings/arxiv-cs-ai-grounded-but-not-immune-confirmation-bias-in-agent-b4157dc3.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-hallucination","name":"Hallucination","explanation":"When an AI system states something false or made up as if it were true.","referenceUrl":"https://developers.google.com/machine-learning/glossary#hallucination"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"},{"id":"claude","name":"Claude","kind":"model","context":"Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.","referenceUrls":["https://claude.com/product/overview","https://www.anthropic.com/news","https://www.anthropic.com/research/team/economics"],"path":"/entities/claude","json":"/entities/claude.json","markdown":"/entities/claude.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"b4157dc37d84a4ed3fe269c947043b87229c4f3487e2dbd1953369e28eefad63","revision":"dc1f9f97-2449-4f70-8c4f-c57477026861","revisionKind":"publication-id","title":"arXiv cs.AI: Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge","url":"https://escapemeridian.com/readings/arxiv-cs-ai-grounded-but-not-immune-confirmation-bias-in-agent-b4157dc3"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.05662","revision":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","revisionKind":"sha256-source-text","title":"Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge","url":"https://arxiv.org/abs/2611.05662"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"b4157dc37d84a4ed3fe269c947043b87229c4f3487e2dbd1953369e28eefad63","title":"arXiv cs.AI: Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge","argument":"The authors of “Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge” argue that an Agent-as-a-Judge can gather decisive evidence and still be swayed by a false note attributed to an authority. Across four judges and five datasets, such a note overturned up to 40% of tool-using judges’ correct verdicts, even when the judges collected decisive evidence. The authors treat the note as a controlled diagnostic of confirmation bias, not an adversarial attack. They identify three ways the shift can happen: over-trust, when a judge verifies evidence and then defers; hallucination, when it fabricates support for changing its verdict; and under-investigation, when it stops at the first corroborating signal. For the two Claude judges, the authors say over-trust accounted for most harmful flips; smaller judges showed all three failure modes. Lightweight mitigations reduced the bias but did not remove it. The result raises a practical concern for people relying on these judges to evaluate open-ended agent tasks: collecting evidence alone may not prevent an irrelevant authority cue from changing a verdict. The abstract does not specify which cue properties drive the shift or how the mitigations were tested.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The source is marked “Preprint. Under Review”; the evidence provided is the abstract, not the full paper. The abstract says the shift is driven by specific properties of the cue but does not identify those properties or describe how the lightweight mitigations were evaluated. The amount by which they reduce bias is not stated.","speaker":{"name":"Brown, Drake Benjamin, Pahwa, Khushbu, Ge, Yubin, Romeo, Salvatore, Agnihotri, Akhil, Zhang, Yi, Ma, Xiaofei, Bonadiman, Daniele","roleAtSource":"Article author"},"source":{"title":"Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.05662","publishedAt":"2026-11-03","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors report that a false pre-screening note attributed to an authority overturned up to 40% of tool-using judges’ correct verdicts across four judges and five datasets, despite decisive evidence collection.","excerpt":"a single false pre-screening note, attributed to an authority and carrying no evidential bearing on the correct verdict, overturns up to 40% of a tool-using judge's correct verdicts across four judges and five datasets, even though the judge collects decisive evidence.","url":"https://arxiv.org/abs/2611.05662","relationship":"Source-stated"},{"text":"The authors identify over-trust, hallucination, and under-investigation as three failure modes through which confirmation bias changes judge behavior.","excerpt":"We classify changes in judge behavior and attribute changes to one of three failure modes: over-trust (verify, conclude correctly, then defer), hallucination (fabricate support for the flip), and under-investigation (stop at the first corroborating signal).","url":"https://arxiv.org/abs/2611.05662","relationship":"Source-stated"},{"text":"The authors say over-trust predominated for the two Claude judges, while smaller judges showed all three error modes; lightweight mitigations blunted but did not remove the bias.","excerpt":"For the two Claude judges, the failure is predominantly over-trust (67-75% of harmful flips), while smaller judges have error modes spread across all three. Lightweight mitigations blunt but never remove the bias.","url":"https://arxiv.org/abs/2611.05662","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05662","checkedAt":"2026-11-09T10:54:09.921Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Which properties of the false note drive the change in verdict, and how much do the lightweight mitigations reduce it?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge","summary":"The authors of “Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge” argue that an Agent-as-a-Judge can gather decisive evidence and still be swayed by a false note attributed to an authority. Across four judges and five datasets, such a note overturned up to 40% of tool-using judges’ correct verdicts, even when the judges collected decisive evidence. The authors treat the note as a controlled diagnostic of confirmation bias, not an adversarial attack. They identify three ways the shift can happen: over-trust, when a judge verifies evidence and then defers; hallucination, when it fabricates support for changing its verdict; and under-investigation, when it stops at the first corroborating signal. For the two Claude judges, the authors say over-trust accounted for most harmful flips; smaller judges showed all three failure modes. Lightweight mitigations reduced the bias but did not remove it. The result raises a practical concern for people relying on these judges to evaluate open-ended agent tasks: collecting evidence alone may not prevent an irrelevant authority cue from changing a verdict. The abstract does not specify which cue properties drive the shift or how the mitigations were tested.","question":"Which properties of the false note drive the change in verdict, and how much do the lightweight mitigations reduce it?","claims":[{"text":"The authors report that a false pre-screening note attributed to an authority overturned up to 40% of tool-using judges’ correct verdicts across four judges and five datasets, despite decisive evidence collection.","excerpt":"a single false pre-screening note, attributed to an authority and carrying no evidential bearing on the correct verdict, overturns up to 40% of a tool-using judge's correct verdicts across four judges and five datasets, even though the judge collects decisive evidence.","support":"source-stated"},{"text":"The authors identify over-trust, hallucination, and under-investigation as three failure modes through which confirmation bias changes judge behavior.","excerpt":"We classify changes in judge behavior and attribute changes to one of three failure modes: over-trust (verify, conclude correctly, then defer), hallucination (fabricate support for the flip), and under-investigation (stop at the first corroborating signal).","support":"source-stated"},{"text":"The authors say over-trust predominated for the two Claude judges, while smaller judges showed all three error modes; lightweight mitigations blunted but did not remove the bias.","excerpt":"For the two Claude judges, the failure is predominantly over-trust (67-75% of harmful flips), while smaller judges have error modes spread across all three. Lightweight mitigations blunt but never remove the bias.","support":"source-stated"}],"limitations":["The source is marked “Preprint. Under Review”; the evidence provided is the abstract, not the full paper.","The abstract says the shift is driven by specific properties of the cue but does not identify those properties or describe how the lightweight mitigations were evaluated. The amount by which they reduce bias is not stated."],"mediaId":"9fdb6f74ae483d9b2394371c90ab1b2a2328540f77d3c7acd7e55bd618f0c82e","correctionReason":null,"id":"b4157dc37d84a4ed3fe269c947043b87229c4f3487e2dbd1953369e28eefad63","publicationId":"dc1f9f97-2449-4f70-8c4f-c57477026861","version":1,"sourceUrl":"https://arxiv.org/abs/2611.05662","sourceTitle":"Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-03","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:06:20.581Z","createdAt":"2026-11-09T10:23:16.555Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"9fdb6f74ae483d9b2394371c90ab1b2a2328540f77d3c7acd7e55bd618f0c82e","kind":"article","url":"https://arxiv.org/abs/2611.05662","sourceUrl":"https://arxiv.org/abs/2611.05662","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T10:22:30.261Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05662","checkedAt":"2026-11-09T10:54:09.921Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"9fdb6f74ae483d9b2394371c90ab1b2a2328540f77d3c7acd7e55bd618f0c82e","confidence":0.84,"provider":"typesafe","model":"jev-1.13.0","inputHash":"84d208ce3aa455428c6faa3e2ba02aae641e2ab01dfe529584316dd3fe8032db","decidedAt":"2026-11-09T10:22:31.385Z"},"provenance":{"textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observationId":"5aef6733a2c742849e31c62e79a6e98fceae5cf8c511bf09e5ec577f2940c2fd","retrievedAt":"2026-11-09T10:22:30.064Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"87f3701f-683e-432e-8ff0-319df89d7166"},"reviewer":{"model":"gpt-6-luna","runId":"e56894bb-ddc4-4499-8f67-2c9da34ded7d"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T10:23:16.555Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Brown, Drake Benjamin","url":null,"kind":"unknown"},{"name":"Pahwa, Khushbu","url":null,"kind":"unknown"},{"name":"Ge, Yubin","url":null,"kind":"unknown"},{"name":"Romeo, Salvatore","url":null,"kind":"unknown"},{"name":"Agnihotri, Akhil","url":null,"kind":"unknown"},{"name":"Zhang, Yi","url":null,"kind":"unknown"},{"name":"Ma, Xiaofei","url":null,"kind":"unknown"},{"name":"Bonadiman, Daniele","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Brown, Drake Benjamin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Brown, Drake Benjamin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Pahwa, Khushbu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Pahwa, Khushbu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Ge, Yubin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ge, Yubin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Romeo, Salvatore","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Romeo, Salvatore","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Agnihotri, Akhil","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Agnihotri, Akhil","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Zhang, Yi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhang, Yi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Ma, Xiaofei","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ma, Xiaofei","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}},{"actor":{"name":"Bonadiman, Daniele","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Bonadiman, Daniele","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05662","locator":"head/meta[name=author|citation_author]"},"assertionId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T10:54:09.921Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","sourceUrl":"https://arxiv.org/abs/2611.05662","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z"}}],"participationCoverage":{"total":8,"returned":8,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.05662","snapshotId":"9a18674ae0cdf0eb5ae980c33dd7fb7a337b15effa55622696cfd52edf782a94","contentSha256":"e5411088d9e3bb9a897686218225760981987229edad4bf810208a66693870a8","textHash":"4981375b641273c9b3a7ce8bde47e22f658403c43eea5f2f636ab38d17c2a008","observedAt":"2026-11-09T10:54:09.921Z","basis":"publisher-page-metadata","sourceLinks":47,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/b4157dc37d84a4ed3fe269c947043b87229c4f3487e2dbd1953369e28eefad63/source"}}}],"audits":[{"publicationId":"dc1f9f97-2449-4f70-8c4f-c57477026861","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The findings, failure-mode taxonomy, Claude-judge breakdown, and mitigation result are attributed to the paper's authors. The source is correctly identified as an arXiv cs.AI preprint, and the 'Preprint. Under Review' status is noted in the limitations.","excerpt":"Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge"},{"dimension":"summary","status":"uncertain","explanation":"The reading is faithful but never says what an Agent-as-a-Judge is: an AI evaluator given tools and another agent's full trajectory so it can check a result against the environment. A general reader may not realize an AI is grading another AI's work, and 'trajectory' and 'tool-using' have no plain gloss. Smallest repair: add one plain sentence defining the term using the abstract's own description, and suggest a term-library tooltip. The title only prefixes the source name, but that is acceptable. The question is substantive.","excerpt":"an Agent-as-a-Judge: an evaluator handed tools and the agent's full trajectory so it can verify a result against the environment"},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-11-03, which matches the arXiv v1 submission date. It is not the later feed date of Nov 5.","excerpt":"[Submitted on 3 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"The claims match the abstract: up to 40% of correct verdicts overturned across four judges and five datasets; the note framed as a diagnostic, not an attack; three failure modes; over-trust predominating (67-75%) for the Claude judges; and mitigations that blunt but never remove the bias. The practical concern is framed as a possibility. The stated gaps about cue properties and mitigation testing are accurate.","excerpt":"Lightweight mitigations blunt but never remove the bias."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source.","excerpt":null}],"createdAt":"2026-11-09T10:23:16.555Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}