Escape Meridian

Xiao, Charles · Nov 16, 2026

arXiv cs.AI: Safe Resemblance, Silent Propagation: Limits of Text-Similarity Guardrails for AI Agent Safety

Markdown · JSON · Original source

Charles Xiao argues that text-similarity guardrails can confuse injected instructions that keep affecting after their wording changes with legitimate quotation and template reuse. In a controlled three-agent email relay, DonkeyRail’s default threshold missed 8 of 19 functional outputs and flagged 70 of 192 legitimate replies; every completed quotation and template-reuse reply triggered a false alert. Calibration for low false positives on one template set did not transfer to new templates: the recalibrated threshold detected only one functional output and still produced 15 false alerts. Xiao also reports an operating-point error in all 21 published classifier configurations: ROC interpolation replaced direct threshold application. More broadly, high AUC against a surface-level text criterion provides no distribution-free guarantee of high AUC against the behavioral target, and existing evaluation labels lack the joint observations needed to estimate that gap. The paper also finds that identical visible transcripts can call for different safety decisions under different governing policies, a distinction no observation-only classifier can resolve. Xiao argues for behavioral and policy-grounded evaluation over text-similarity guardrails.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

arXiv cs.AIArticle · arxiv.orgOpen original source

Original source

cs.AI · Article

Source details

Original source date: Nov 16, 2026 (2026-11-16).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-17T13:47:12.534Z. First observed: 2026-11-17T08:27:46.537Z. Retrieved: 2026-11-17T13:34:47.814Z.

Referenced by

Recorded citations in public readings. A reference alone does not establish agreement or influence.

No public references recorded here yet.

Reference data (JSON)

Limits

The reported classifier results are from a controlled three-agent email relay using DonkeyRail; the abstract does not establish how they transfer to other settings. The abstract says existing evaluation labels lack the joint observations needed to estimate the gap between the surface-level text criterion and the behavioral target.

Open question

What joint observations would be needed to estimate the gap between performance against a surface-level text criterion and performance against the behavioral target?

Source-stated

Xiao argues that text-similarity guardrails risk confusing injected instructions that retain their effect after rewording with legitimate quotation and template reuse.

an output can preserve an injected instruction's effect on downstream agents even after its wording changes, yet legitimate quotation and template reuse can produce high textual overlap with the source

Source-stated

In the controlled three-agent email relay, DonkeyRail’s default threshold missed 8 functional outputs and flagged 70 legitimate replies.

At its default threshold the classifier misses 8 of 19 functional outputs while flagging 70 of 192 legitimate replies

Source-stated

The paper reports that recalibration on one template set did not transfer to new templates, and that the existing evaluation labels cannot estimate the gap between the surface-level text criterion and behavioral target.

the recalibrated threshold detects only one functional output and still produces 15 false alerts on new templates

Source-stated

Xiao argues that identical visible transcripts can require different safety decisions under governing policies, beyond the reach of observation-only classifiers.

identical visible transcripts can require different safety decisions depending on the governing policy, a distinction no observation-only classifier can resolve

Source and contributor context

Xiao, Charles · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

No suitable media was found by the supported page extraction.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

AI agent

An AI system that chooses and carries out actions toward a goal, often using software tools.

arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

Editorial provenance

Version 1. Publication ID: 47b63d03-acff-45ca-8bcb-c8cbf7c33220.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-17T13:47:12.534Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material_development.

Observation: 06b56ee55a2e3773d04a679282a2db0530d48c65f78526599e3193786f71df61. Source text hash: 76289b5d3bf9d5c0e073beb36000310050320680273d5736bc963b884214fa1e.

Correction reason: Restored that calibration for low false positives on one template set did not transfer, clarifying the scope of the reported result.

Revision and source-check history

Version 1 · 2026-11-17T13:47:12.534Z · Restored that calibration for low false positives on one template set did not transfer, clarifying the scope of the reported result.

arXiv cs.AI: Safe Resemblance, Silent Propagation: Limits of Text-Similarity Guardrails for AI Agent Safety

Charles Xiao argues that text-similarity guardrails can confuse injected instructions that keep affecting after their wording changes with legitimate quotation and template reuse. In a controlled three-agent email relay, DonkeyRail’s default threshold missed 8 of 19 functional outputs and flagged 70 of 192 legitimate replies; every completed quotation and template-reuse reply triggered a false alert. Calibration for low false positives on one template set did not transfer to new templates: the recalibrated threshold detected only one functional output and still produced 15 false alerts. Xiao also reports an operating-point error in all 21 published classifier configurations: ROC interpolation replaced direct threshold application. More broadly, high AUC against a surface-level text criterion provides no distribution-free guarantee of high AUC against the behavioral target, and existing evaluation labels lack the joint observations needed to estimate that gap. The paper also finds that identical visible transcripts can call for different safety decisions under different governing policies, a distinction no observation-only classifier can resolve. Xiao argues for behavioral and policy-grounded evaluation over text-similarity guardrails.

Original source

cs.AI · Article

Source details

Original source date: Nov 16, 2026 (2026-11-16).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-17T13:47:12.534Z. First observed: 2026-11-17T08:27:46.537Z. Retrieved: 2026-11-17T13:34:47.814Z.

Limits

The reported classifier results are from a controlled three-agent email relay using DonkeyRail; the abstract does not establish how they transfer to other settings. The abstract says existing evaluation labels lack the joint observations needed to estimate the gap between the surface-level text criterion and the behavioral target.

Open question

What joint observations would be needed to estimate the gap between performance against a surface-level text criterion and performance against the behavioral target?

Source-stated

Xiao argues that text-similarity guardrails risk confusing injected instructions that retain their effect after rewording with legitimate quotation and template reuse.

an output can preserve an injected instruction's effect on downstream agents even after its wording changes, yet legitimate quotation and template reuse can produce high textual overlap with the source

Source-stated

In the controlled three-agent email relay, DonkeyRail’s default threshold missed 8 functional outputs and flagged 70 legitimate replies.

At its default threshold the classifier misses 8 of 19 functional outputs while flagging 70 of 192 legitimate replies

Source-stated

The paper reports that recalibration on one template set did not transfer to new templates, and that the existing evaluation labels cannot estimate the gap between the surface-level text criterion and behavioral target.

the recalibrated threshold detects only one functional output and still produces 15 false alerts on new templates

Source-stated

Xiao argues that identical visible transcripts can require different safety decisions under governing policies, beyond the reach of observation-only classifiers.

identical visible transcripts can require different safety decisions depending on the governing policy, a distinction no observation-only classifier can resolve

Source and contributor context

Xiao, Charles · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

No suitable media was found by the supported page extraction.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

AI agent

An AI system that chooses and carries out actions toward a goal, often using software tools.

arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

Editorial provenance

Version 1. Publication ID: 47b63d03-acff-45ca-8bcb-c8cbf7c33220.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-17T13:47:12.534Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material_development.

Observation: 06b56ee55a2e3773d04a679282a2db0530d48c65f78526599e3193786f71df61. Source text hash: 76289b5d3bf9d5c0e073beb36000310050320680273d5736bc963b884214fa1e.

Correction reason: Restored that calibration for low false positives on one template set did not transfer, clarifying the scope of the reported result.

uncertain · 2026-11-17T13:47:12.534Z · anthropic/claude-opus-5.5

Publication: 47b63d03-acff-45ca-8bcb-c8cbf7c33220

attribution — uncertain: The reading correctly credits Charles Xiao and arXiv cs.AI. However, the sentence beginning 'More broadly, high AUC against a surface-level text criterion provides no distribution-free guarantee...' is written as an established fact. In the source, it is the author's formal argument. Smallest repair: 'Xiao argues formally that high AUC...'

show formally that high AUC against a surface-level text criterion provides no distribution-free guarantee of high AUC against the behavioral target

summary — uncertain: The numbers are faithful, but a general reader cannot follow the central meaning. 'Functional outputs' is the key measure and is never explained; the paper defines it as an injected procedure that makes two later agents run fresh computations. Several other terms are also left unexplained, including injected instruction, AUC, ROC interpolation, operating-point error and distribution-free guarantee. The reading never states plainly why the results matter: a guardrail that relies on text overlap can let a reworded attack through while flagging legitimate quoting. The title simply repeats the source title. Smallest repair: add a plain-language definition of functional outputs and a one-line significance sentence.

an output counts as functional (the injected procedure induces fresh computations in two subsequent agents) independent of the detector's score

date — pass: The originalPublished value, 2026-11-16, matches the v1 submission date shown on the arXiv page. It is not the later feed date.

[Submitted on 16 Nov 2026]

support — pass: All figures match the abstract: 8 of 19 missed, 70 of 192 flagged, 15 false alerts, one detection after recalibration, and 21 configurations. The policy-transcript claim and the conclusion are also supported. The question follows directly from the stated gap in evaluation labels and does not presuppose an answer. The limitations are appropriately scoped.

the existing evaluation labels lack the joint observations needed to estimate the gap

media — pass: The selected media is the verified direct link to the arXiv abstract page, which is the source itself.

Challenge or correct this reading

Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.