Weng, Shihao, Feng, Yang, Xie, Xiaofei · May 7, 2026
arXiv cs.AI paper stress-tests whether LLM safety judges are policy-invariant
Markdown · JSON · Original source
Shihao Weng, Yang Feng and Xiaofei Xie argue that a safety judge should assess an agent’s behavior consistently when evaluation policies are rewritten without changing their meaning. They call this property policy invariance and test it through three principles: rubric-semantics invariance, rubric-threshold invariance and ambiguity-aware calibration. In tests using four agent-class judges and trajectories from ASSEBench and R-Judge, the authors report that judges reacted comparably to meaningful policy shifts and meaning-preserving rewrites, without distinguishing them. Such rewrites flipped up to 9.1% of verdicts above baseline jitter; 18–43% of observed flips occurred on cases the authors classify as unambiguous. The authors say the findings also held with a frontier-scale judge, a third and a second base policy. Their concern is that safety scores can partly reflect how an evaluator is prompted, rather than only what an agent did. They introduce the Policy Invariance Score and Judge Card reporting protocol, which they say expose a wide spread in judge reliability that accuracy-only leaderboards miss. The abstract describes a stress-test and reports its results; it does not establish how often these effects occur in deployed safety evaluations.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: May 7, 2026 (2026-05-07).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-13T21:33:54.045Z. First observed: 2026-11-06T05:07:04.170Z. Retrieved: 2026-11-13T21:32:40.439Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The evidence provided is the paper’s abstract, which does not give detailed methods or results for individual judges, or policy rewrites. The abstract reports stress-test results but does not establish how often the reported effects occur in deployed safety evaluations.
Open question
How do the authors’ stress-test results vary across the judges, and policy rewrites they tested?
Source-stated
The authors argue that trustworthy safety judges should be policy-invariant and set out three principles for testing that property.
any trustworthy safety judge must satisfy a basic property we call policy invariance, and we operationalize it as three testable principles
Source-stated
In the reported tests, judges responded similarly to meaningful policy shifts and meaning-preserving structural rewrites, and did not distinguish the two.
today's judges respond to meaningful normative shifts and to meaningless structural rewrites with comparable strength, and cannot tell the two apart.
Source-stated
The authors report that content-preserving rewrites flipped up to 9.1% of verdicts above baseline jitter, and that 18–43% of observed flips occurred on unambiguous cases.
Content-preserving policy rewrites flip up to 9.1% of verdicts above baseline jitter, and 18-43% of all observed flips occur on unambiguous cases under such rewrites
Source-stated
The authors say their findings persisted with a frontier-scale judge, a third benchmark and a second base policy.
These findings persist with a frontier-scale judge, a third benchmark, and a second base policy.
Source-stated
The authors say their Policy Invariance Score and Judge Card reporting protocol expose a spread in judge reliability that accuracy-only leaderboards do not show.
we contribute the Policy Invariance Score and the Judge Card reporting protocol, which expose an order-of-magnitude spread in judge reliability that is invisible to accuracy-only leaderboards.
Source and contributor context
Weng, Shihao, Feng, Yang, Xie, Xiaofei · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 34824a32-f0dc-4530-8d30-55f376577a9c.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T21:33:54.045Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 68c300435d830bea184615f2cf89a9313ac71dd4ead2cceaf516355f24b4fc07. Source text hash: 6b5306566b102f4d753fe302b9844912a9717d5f2cb2fcf76cb4b8a8a97aae22.
Revision and source-check history
Version 1 · 2026-11-13T21:33:54.045Z
arXiv cs.AI paper stress-tests whether LLM safety judges are policy-invariant
Shihao Weng, Yang Feng and Xiaofei Xie argue that a safety judge should assess an agent’s behavior consistently when evaluation policies are rewritten without changing their meaning. They call this property policy invariance and test it through three principles: rubric-semantics invariance, rubric-threshold invariance and ambiguity-aware calibration. In tests using four agent-class judges and trajectories from ASSEBench and R-Judge, the authors report that judges reacted comparably to meaningful policy shifts and meaning-preserving rewrites, without distinguishing them. Such rewrites flipped up to 9.1% of verdicts above baseline jitter; 18–43% of observed flips occurred on cases the authors classify as unambiguous. The authors say the findings also held with a frontier-scale judge, a third and a second base policy. Their concern is that safety scores can partly reflect how an evaluator is prompted, rather than only what an agent did. They introduce the Policy Invariance Score and Judge Card reporting protocol, which they say expose a wide spread in judge reliability that accuracy-only leaderboards miss. The abstract describes a stress-test and reports its results; it does not establish how often these effects occur in deployed safety evaluations.
Original source
cs.AI · Article
Source details
Original source date: May 7, 2026 (2026-05-07).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-13T21:33:54.045Z. First observed: 2026-11-06T05:07:04.170Z. Retrieved: 2026-11-13T21:32:40.439Z.
Limits
The evidence provided is the paper’s abstract, which does not give detailed methods or results for individual judges, or policy rewrites. The abstract reports stress-test results but does not establish how often the reported effects occur in deployed safety evaluations.
Open question
How do the authors’ stress-test results vary across the judges, and policy rewrites they tested?
Source-stated
The authors argue that trustworthy safety judges should be policy-invariant and set out three principles for testing that property.
any trustworthy safety judge must satisfy a basic property we call policy invariance, and we operationalize it as three testable principles
Source-stated
In the reported tests, judges responded similarly to meaningful policy shifts and meaning-preserving structural rewrites, and did not distinguish the two.
today's judges respond to meaningful normative shifts and to meaningless structural rewrites with comparable strength, and cannot tell the two apart.
Source-stated
The authors report that content-preserving rewrites flipped up to 9.1% of verdicts above baseline jitter, and that 18–43% of observed flips occurred on unambiguous cases.
Content-preserving policy rewrites flip up to 9.1% of verdicts above baseline jitter, and 18-43% of all observed flips occur on unambiguous cases under such rewrites
Source-stated
The authors say their findings persisted with a frontier-scale judge, a third benchmark and a second base policy.
These findings persist with a frontier-scale judge, a third benchmark, and a second base policy.
Source-stated
The authors say their Policy Invariance Score and Judge Card reporting protocol expose a spread in judge reliability that accuracy-only leaderboards do not show.
we contribute the Policy Invariance Score and the Judge Card reporting protocol, which expose an order-of-magnitude spread in judge reliability that is invisible to accuracy-only leaderboards.
Source and contributor context
Weng, Shihao, Feng, Yang, Xie, Xiaofei · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 34824a32-f0dc-4530-8d30-55f376577a9c.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T21:33:54.045Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 68c300435d830bea184615f2cf89a9313ac71dd4ead2cceaf516355f24b4fc07. Source text hash: 6b5306566b102f4d753fe302b9844912a9717d5f2cb2fcf76cb4b8a8a97aae22.
uncertain · 2026-11-13T21:33:54.045Z · anthropic/claude-opus-5.5
Publication: 34824a32-f0dc-4530-8d30-55f376577a9c
attribution — pass: The reading attributes the paper to Shihao Weng, Yang Feng and Xiaofei Xie, matching the arXiv listing. It keeps all findings framed as the authors' arguments and reported results.
Authors:Shihao Weng, Yang Feng, Xiaofei Xie
summary — uncertain: The facts are faithful, but a general reader is left without key context. The reading never says the 'safety judge' is itself an LLM prompted to grade agent safety, which the source frames as the de facto evaluator. 'Agent-class judges', 'trajectories', 'baseline jitter' and 'frontier-scale' are unexplained. The three principles are only named; the threshold principle (judges should respond to intentional strict-to-lenient shifts) is not explained. 'Order-of-magnitude spread' is softened to 'wide spread'. Smallest repair: add tooltips or brief glosses for these terms and the LLM-as-a-Judge context, and restore 'order-of-magnitude'.
LLM-as-a-Judge pipelines have become the de facto evaluator for agent safety
date — pass: The displayed date of 2026-05-07 matches the v1 submission date. It correctly ignores the v2 revision of 5 Nov 2026 and the November feed date.
Submitted on 7 May 2026 (v1), last revised 5 Nov 2026 (this version, v2)
support — pass: Each claim is directly supported by a verbatim excerpt. The figures (9.1%, 18-43%), the robustness checks, and the concern that scores conflate agent behavior with evaluator prompting all match the abstract. The limitations correctly note that the abstract does not establish how often these effects occur in deployed evaluations. The question is a genuine open issue rather than a leading claim.
so safety scores produced by these judges partly conflate what the agent did with how the evaluator was prompted
media — pass: The selected media is the verified direct link to the original arXiv abstract page, which is the source itself.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference