# arXiv cs.AI: Measuring Prompt Injection Guards on Legal Contract Review Attacks [Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-measuring-prompt-injection-guards-on-legal-contract-8d6a3f25) Mark Heimann, Katherine Taylor and Bruno Jacob report that specialized prompt injection guards tested on legal contracts can miss attacks and misclassify ordinary document content. In their evaluation of eleven open-weights guards across two contract corpora, document-level false positive rates reached 92.39%; two guards scored below chance on paired AUC, and a five-regex baseline was not Pareto-dominated by any guard they tested. The authors say controlled probes exposed both model-specific and general failure modes. Prompted LLMs resolved the tested failures, including unauthorized scope claims, a threat category on which every guard scored zero. They also report that a cascaded LLM solution—using a mid-tier model and having a frontier model verify its positive results—cost 6.5x less than using only a frontier model while approaching its performance. Escape Meridian interpretation: the reported false positives make visible a practical tension: guards intended to protect document analysis systems may themselves flag contract content, while the proposed cascade adds a verification step and reports lower cost. The abstract reports these results but does not provide the evaluation details needed to assess how the findings might carry over to other contracts or systems. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 6, 2026 (2026-11-06). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-09T06:46:33.176Z. First observed: 2026-11-09T05:03:46.697Z. Retrieved: 2026-11-09T06:16:46.125Z. - [Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks](https://arxiv.org/abs/2611.08834) ## Limits The supplied evidence is the paper's abstract; it does not include the evaluation methods, corpus details, or results needed to independently assess the reported findings. The abstract reports results for two corpora of legal contracts and the evaluated guards; it does not establish how those results generalize to other contracts or systems. ## Open question What evaluation details would help determine whether the reported guard and cascade results carry over beyond the two contract corpora and systems tested? ## Source-stated The authors evaluated eleven open-weights guards on two corpora of legal contracts. > We measure eleven open-weights guards on two corpora of legal contracts - [Supporting source](https://arxiv.org/abs/2611.08834) ## Source-stated The authors report document-level false positive rates as high as 92.39%, and two guards fell below chance on paired AUC. > document-level false positive rates are as high as 92.39%; two guards fall below chance on paired AUC - [Supporting source](https://arxiv.org/abs/2611.08834) ## Source-stated A five-regex baseline was not Pareto-dominated by any guard in the panel. > a five-regex baseline is not Pareto-dominated by any guard in the panel. - [Supporting source](https://arxiv.org/abs/2611.08834) ## Source-stated The authors say prompted LLMs resolved the tested failures, including unauthorized scope claims, where every guard scored zero. > including a second threat category (unauthorized scope claims) on which every guard scores zero. - [Supporting source](https://arxiv.org/abs/2611.08834) ## Source-stated The authors report that a cascade using a mid-tier model and a frontier model to verify its positive results costs 6.5x less than using only a frontier model, while approaching its performance. > A cascaded LLM solution, where the positives of a mid-tier model are verified by a frontier model, is 6.5x cheaper than using exclusively a frontier model and approaches its performance. - [Supporting source](https://arxiv.org/abs/2611.08834) ## Escape Meridian interpretation The abstract does not provide enough evaluation detail to assess how the findings carry over to other contracts or systems. > We measure eleven open-weights guards on two corpora of legal contracts - [Supporting source](https://arxiv.org/abs/2611.08834) ## Source and contributor context Heimann, Mark, Taylor, Katherine, Jacob, Bruno · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Prompt injection Hidden instructions placed in a document, email or web page that try to take control of an AI system reading it. - [Definition reference](https://en.wikipedia.org/wiki/Prompt_injection) ## Open weights A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted. - [Definition reference](https://opensource.org/ai/open-weights) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## Frontier model One of the most capable AI models available at a given time; the set shifts as models improve. - [Definition reference](https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 0bba5917-0aa2-49b4-832b-e6a04df62bf6. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T06:46:33.176Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 52e84440789f91d5edaecb2684275c079244da30fd6072eaa4675c136256f0a2. Source text hash: 9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d. ## Revision and source-check history ### Version 1 — 2026-11-09T06:46:33.176Z Mark Heimann, Katherine Taylor and Bruno Jacob report that specialized prompt injection guards tested on legal contracts can miss attacks and misclassify ordinary document content. In their evaluation of eleven open-weights guards across two contract corpora, document-level false positive rates reached 92.39%; two guards scored below chance on paired AUC, and a five-regex baseline was not Pareto-dominated by any guard they tested. The authors say controlled probes exposed both model-specific and general failure modes. Prompted LLMs resolved the tested failures, including unauthorized scope claims, a threat category on which every guard scored zero. They also report that a cascaded LLM solution—using a mid-tier model and having a frontier model verify its positive results—cost 6.5x less than using only a frontier model while approaching its performance. Escape Meridian interpretation: the reported false positives make visible a practical tension: guards intended to protect document analysis systems may themselves flag contract content, while the proposed cascade adds a verification step and reports lower cost. The abstract reports these results but does not provide the evaluation details needed to assess how the findings might carry over to other contracts or systems. ## Original source arXiv cs.AI · Article Original source date: Nov 6, 2026 (2026-11-06). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-09T06:46:33.176Z. First observed: 2026-11-09T05:03:46.697Z. Retrieved: 2026-11-09T06:16:46.125Z. - [Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks](https://arxiv.org/abs/2611.08834) ## Limits The supplied evidence is the paper's abstract; it does not include the evaluation methods, corpus details, or results needed to independently assess the reported findings. The abstract reports results for two corpora of legal contracts and the evaluated guards; it does not establish how those results generalize to other contracts or systems. ## Open question What evaluation details would help determine whether the reported guard and cascade results carry over beyond the two contract corpora and systems tested? ## Source-stated The authors evaluated eleven open-weights guards on two corpora of legal contracts. > We measure eleven open-weights guards on two corpora of legal contracts - [Supporting source](https://arxiv.org/abs/2611.08834) ## Source-stated The authors report document-level false positive rates as high as 92.39%, and two guards fell below chance on paired AUC. > document-level false positive rates are as high as 92.39%; two guards fall below chance on paired AUC - [Supporting source](https://arxiv.org/abs/2611.08834) ## Source-stated A five-regex baseline was not Pareto-dominated by any guard in the panel. > a five-regex baseline is not Pareto-dominated by any guard in the panel. - [Supporting source](https://arxiv.org/abs/2611.08834) ## Source-stated The authors say prompted LLMs resolved the tested failures, including unauthorized scope claims, where every guard scored zero. > including a second threat category (unauthorized scope claims) on which every guard scores zero. - [Supporting source](https://arxiv.org/abs/2611.08834) ## Source-stated The authors report that a cascade using a mid-tier model and a frontier model to verify its positive results costs 6.5x less than using only a frontier model, while approaching its performance. > A cascaded LLM solution, where the positives of a mid-tier model are verified by a frontier model, is 6.5x cheaper than using exclusively a frontier model and approaches its performance. - [Supporting source](https://arxiv.org/abs/2611.08834) ## Escape Meridian interpretation The abstract does not provide enough evaluation detail to assess how the findings carry over to other contracts or systems. > We measure eleven open-weights guards on two corpora of legal contracts - [Supporting source](https://arxiv.org/abs/2611.08834) ## Source and contributor context Heimann, Mark, Taylor, Katherine, Jacob, Bruno · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Prompt injection Hidden instructions placed in a document, email or web page that try to take control of an AI system reading it. - [Definition reference](https://en.wikipedia.org/wiki/Prompt_injection) ## Open weights A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted. - [Definition reference](https://opensource.org/ai/open-weights) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## Frontier model One of the most capable AI models available at a given time; the set shifts as models improve. - [Definition reference](https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 0bba5917-0aa2-49b4-832b-e6a04df62bf6. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T06:46:33.176Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 52e84440789f91d5edaecb2684275c079244da30fd6072eaa4675c136256f0a2. Source text hash: 9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d. ### uncertain — 2026-11-09T06:46:33.176Z — anthropic/claude-opus-5.5 Publication: 0bba5917-0aa2-49b4-832b-e6a04df62bf6 attribution — pass: The draft correctly names authors Mark Heimann, Katherine Taylor and Bruno Jacob. It frames the findings as their reported results, and labels the closing inference as Escape Meridian interpretation. 'Prompted LLMs resolved the tested failures' is not explicitly attributed, but it follows 'The authors say' and stays within the abstract's reporting frame. > Authors:Mark Heimann, Katherine Taylor, Bruno Jacob summary — uncertain: The numbers and findings are faithful, but central terms go unexplained for a non-specialist: prompt injection guard, open-weights, paired AUC, Pareto-dominated, regex and frontier model. The paper's notable point is that a simple five-regex baseline is not beaten on both error trade-offs by any guard. That point is hidden behind 'Pareto-dominated'. Smallest repair: one plain sentence saying a prompt injection guard screens documents for hidden instructions aimed at an AI. Add another saying no guard beat a simple five-pattern text filter on both catching attacks and avoiding false alarms. Tooltips could also cover these terms. > a five-regex baseline is not Pareto-dominated by any guard in the panel date — pass: originalPublished is 2026-11-06. That matches the arXiv v1 submission date, not the 2026-11-09 feed date. > \[Submitted on 6 Nov 2026\] support — pass: All reported figures match the abstract: eleven guards, two corpora, 92.39% FPR, below-chance paired AUC, the regex baseline, zero scores on unauthorized scope claims, and the 6.5x cheaper cascade that approaches frontier performance. 'Can miss attacks' is supported by the every-guard-zero result. The labeled interpretation about guards flagging contract content follows from the FPR finding. The limitation and question stay conditional and do not claim real-world reliability. > Prompted LLMs resolve the tested failures, including a second threat category (unauthorized scope claims) on which every guard scores zero. media — pass: The selected media is the verified direct arXiv abstract page for this paper. It is the most direct access to the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/8d6a3f250cbef27e45979f9a797fb394df1be103230adfed6d2e320b381475ef/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-measuring-prompt-injection-guards-on-legal-contract-8d6a3f25.json)