{"id":"8d6a3f250cbef27e45979f9a797fb394df1be103230adfed6d2e320b381475ef","title":"arXiv cs.AI: Measuring Prompt Injection Guards on Legal Contract Review Attacks","argument":"Mark Heimann, Katherine Taylor and Bruno Jacob report that specialized prompt injection guards tested on legal contracts can miss attacks and misclassify ordinary document content. In their evaluation of eleven open-weights guards across two contract corpora, document-level false positive rates reached 92.39%; two guards scored below chance on paired AUC, and a five-regex baseline was not Pareto-dominated by any guard they tested. The authors say controlled probes exposed both model-specific and general failure modes. Prompted LLMs resolved the tested failures, including unauthorized scope claims, a threat category on which every guard scored zero. They also report that a cascaded LLM solution—using a mid-tier model and having a frontier model verify its positive results—cost 6.5x less than using only a frontier model while approaching its performance. Escape Meridian interpretation: the reported false positives make visible a practical tension: guards intended to protect document analysis systems may themselves flag contract content, while the proposed cascade adds a verification step and reports lower cost. The abstract reports these results but does not provide the evaluation details needed to assess how the findings might carry over to other contracts or systems.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the paper's abstract; it does not include the evaluation methods, corpus details, or results needed to independently assess the reported findings. The abstract reports results for two corpora of legal contracts and the evaluated guards; it does not establish how those results generalize to other contracts or systems.","speaker":{"name":"Heimann, Mark, Taylor, Katherine, Jacob, Bruno","roleAtSource":"Article author"},"source":{"title":"Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.08834","publishedAt":"2026-11-06","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors evaluated eleven open-weights guards on two corpora of legal contracts.","excerpt":"We measure eleven open-weights guards on two corpora of legal contracts","url":"https://arxiv.org/abs/2611.08834","relationship":"Source-stated"},{"text":"The authors report document-level false positive rates as high as 92.39%, and two guards fell below chance on paired AUC.","excerpt":"document-level false positive rates are as high as 92.39%; two guards fall below chance on paired AUC","url":"https://arxiv.org/abs/2611.08834","relationship":"Source-stated"},{"text":"A five-regex baseline was not Pareto-dominated by any guard in the panel.","excerpt":"a five-regex baseline is not Pareto-dominated by any guard in the panel.","url":"https://arxiv.org/abs/2611.08834","relationship":"Source-stated"},{"text":"The authors say prompted LLMs resolved the tested failures, including unauthorized scope claims, where every guard scored zero.","excerpt":"including a second threat category (unauthorized scope claims) on which every guard scores zero.","url":"https://arxiv.org/abs/2611.08834","relationship":"Source-stated"},{"text":"The authors report that a cascade using a mid-tier model and a frontier model to verify its positive results costs 6.5x less than using only a frontier model, while approaching its performance.","excerpt":"A cascaded LLM solution, where the positives of a mid-tier model are verified by a frontier model, is 6.5x cheaper than using exclusively a frontier model and approaches its performance.","url":"https://arxiv.org/abs/2611.08834","relationship":"Source-stated"},{"text":"The abstract does not provide enough evaluation detail to assess how the findings carry over to other contracts or systems.","excerpt":"We measure eleven open-weights guards on two corpora of legal contracts","url":"https://arxiv.org/abs/2611.08834","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.08834","checkedAt":"2026-11-09T07:50:08.143Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What evaluation details would help determine whether the reported guard and cascade results carry over beyond the two contract corpora and systems tested?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Measuring Prompt Injection Guards on Legal Contract Review Attacks","summary":"Mark Heimann, Katherine Taylor and Bruno Jacob report that specialized prompt injection guards tested on legal contracts can miss attacks and misclassify ordinary document content. In their evaluation of eleven open-weights guards across two contract corpora, document-level false positive rates reached 92.39%; two guards scored below chance on paired AUC, and a five-regex baseline was not Pareto-dominated by any guard they tested. The authors say controlled probes exposed both model-specific and general failure modes. Prompted LLMs resolved the tested failures, including unauthorized scope claims, a threat category on which every guard scored zero. They also report that a cascaded LLM solution—using a mid-tier model and having a frontier model verify its positive results—cost 6.5x less than using only a frontier model while approaching its performance. Escape Meridian interpretation: the reported false positives make visible a practical tension: guards intended to protect document analysis systems may themselves flag contract content, while the proposed cascade adds a verification step and reports lower cost. The abstract reports these results but does not provide the evaluation details needed to assess how the findings might carry over to other contracts or systems.","question":"What evaluation details would help determine whether the reported guard and cascade results carry over beyond the two contract corpora and systems tested?","claims":[{"text":"The authors evaluated eleven open-weights guards on two corpora of legal contracts.","excerpt":"We measure eleven open-weights guards on two corpora of legal contracts","support":"source-stated"},{"text":"The authors report document-level false positive rates as high as 92.39%, and two guards fell below chance on paired AUC.","excerpt":"document-level false positive rates are as high as 92.39%; two guards fall below chance on paired AUC","support":"source-stated"},{"text":"A five-regex baseline was not Pareto-dominated by any guard in the panel.","excerpt":"a five-regex baseline is not Pareto-dominated by any guard in the panel.","support":"source-stated"},{"text":"The authors say prompted LLMs resolved the tested failures, including unauthorized scope claims, where every guard scored zero.","excerpt":"including a second threat category (unauthorized scope claims) on which every guard scores zero.","support":"source-stated"},{"text":"The authors report that a cascade using a mid-tier model and a frontier model to verify its positive results costs 6.5x less than using only a frontier model, while approaching its performance.","excerpt":"A cascaded LLM solution, where the positives of a mid-tier model are verified by a frontier model, is 6.5x cheaper than using exclusively a frontier model and approaches its performance.","support":"source-stated"},{"text":"The abstract does not provide enough evaluation detail to assess how the findings carry over to other contracts or systems.","excerpt":"We measure eleven open-weights guards on two corpora of legal contracts","support":"interpretation"}],"limitations":["The supplied evidence is the paper's abstract; it does not include the evaluation methods, corpus details, or results needed to independently assess the reported findings.","The abstract reports results for two corpora of legal contracts and the evaluated guards; it does not establish how those results generalize to other contracts or systems."],"mediaId":"0fb5fb29201a559a9abfcdee1eebffb647b2fb07faa4d12ceb2bb2b86c5903b2","correctionReason":null,"id":"8d6a3f250cbef27e45979f9a797fb394df1be103230adfed6d2e320b381475ef","publicationId":"0bba5917-0aa2-49b4-832b-e6a04df62bf6","version":1,"sourceUrl":"https://arxiv.org/abs/2611.08834","sourceTitle":"Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-06","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-09T05:03:46.697Z","createdAt":"2026-11-09T06:46:33.176Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"0fb5fb29201a559a9abfcdee1eebffb647b2fb07faa4d12ceb2bb2b86c5903b2","kind":"article","url":"https://arxiv.org/abs/2611.08834","sourceUrl":"https://arxiv.org/abs/2611.08834","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T06:16:46.410Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.08834","checkedAt":"2026-11-09T07:50:08.143Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"0fb5fb29201a559a9abfcdee1eebffb647b2fb07faa4d12ceb2bb2b86c5903b2","confidence":0.92,"provider":"typesafe","model":"jev-1.13.0","inputHash":"772a66c688a770c48d91b468b7047c9ff9c26eceb992a77cf98b5bfe3db38c79","decidedAt":"2026-11-09T06:16:48.309Z"},"provenance":{"textHash":"9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d","observationId":"52e84440789f91d5edaecb2684275c079244da30fd6072eaa4675c136256f0a2","retrievedAt":"2026-11-09T06:16:46.125Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"21261a93-c3f2-4396-88f5-00aeed020600"},"reviewer":{"model":"gpt-6-luna","runId":"227386ae-94aa-406a-89fc-59adae26c837"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T06:46:33.176Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Heimann, Mark","url":null,"kind":"unknown"},{"name":"Taylor, Katherine","url":null,"kind":"unknown"},{"name":"Jacob, Bruno","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Heimann, Mark","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Heimann, Mark","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08834","locator":"head/meta[name=author|citation_author]"},"assertionId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:50:08.143Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2","sourceUrl":"https://arxiv.org/abs/2611.08834","contentSha256":"244679eb1a6233c22f1d2aaef73ca77c43fc70423e5a1bde8e846f59607e0a78","textHash":"9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d","observedAt":"2026-11-09T07:50:08.143Z"}},{"actor":{"name":"Taylor, Katherine","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Taylor, Katherine","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08834","locator":"head/meta[name=author|citation_author]"},"assertionId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:50:08.143Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2","sourceUrl":"https://arxiv.org/abs/2611.08834","contentSha256":"244679eb1a6233c22f1d2aaef73ca77c43fc70423e5a1bde8e846f59607e0a78","textHash":"9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d","observedAt":"2026-11-09T07:50:08.143Z"}},{"actor":{"name":"Jacob, Bruno","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jacob, Bruno","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08834","locator":"head/meta[name=author|citation_author]"},"assertionId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:50:08.143Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2","sourceUrl":"https://arxiv.org/abs/2611.08834","contentSha256":"244679eb1a6233c22f1d2aaef73ca77c43fc70423e5a1bde8e846f59607e0a78","textHash":"9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d","observedAt":"2026-11-09T07:50:08.143Z"}}],"participationCoverage":{"total":3,"returned":3,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.08834","snapshotId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2","contentSha256":"244679eb1a6233c22f1d2aaef73ca77c43fc70423e5a1bde8e846f59607e0a78","textHash":"9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d","observedAt":"2026-11-09T07:50:08.143Z","basis":"publisher-page-metadata","sourceLinks":41,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/8d6a3f250cbef27e45979f9a797fb394df1be103230adfed6d2e320b381475ef/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-measuring-prompt-injection-guards-on-legal-contract-8d6a3f25","formats":{"html":"/readings/arxiv-cs-ai-measuring-prompt-injection-guards-on-legal-contract-8d6a3f25","markdown":"/readings/arxiv-cs-ai-measuring-prompt-injection-guards-on-legal-contract-8d6a3f25.md","json":"/readings/arxiv-cs-ai-measuring-prompt-injection-guards-on-legal-contract-8d6a3f25.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-prompt-injection","name":"Prompt injection","explanation":"Hidden instructions placed in a document, email or web page that try to take control of an AI system reading it.","referenceUrl":"https://en.wikipedia.org/wiki/Prompt_injection"},{"id":"term-open-weights","name":"Open weights","explanation":"A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.","referenceUrl":"https://opensource.org/ai/open-weights"},{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"},{"id":"term-frontier-model","name":"Frontier model","explanation":"One of the most capable AI models available at a given time; the set shifts as models improve.","referenceUrl":"https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"8d6a3f250cbef27e45979f9a797fb394df1be103230adfed6d2e320b381475ef","revision":"0bba5917-0aa2-49b4-832b-e6a04df62bf6","revisionKind":"publication-id","title":"arXiv cs.AI: Measuring Prompt Injection Guards on Legal Contract Review Attacks","url":"https://escapemeridian.com/readings/arxiv-cs-ai-measuring-prompt-injection-guards-on-legal-contract-8d6a3f25"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.08834","revision":"9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d","revisionKind":"sha256-source-text","title":"Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks","url":"https://arxiv.org/abs/2611.08834"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"8d6a3f250cbef27e45979f9a797fb394df1be103230adfed6d2e320b381475ef","title":"arXiv cs.AI: Measuring Prompt Injection Guards on Legal Contract Review Attacks","argument":"Mark Heimann, Katherine Taylor and Bruno Jacob report that specialized prompt injection guards tested on legal contracts can miss attacks and misclassify ordinary document content. In their evaluation of eleven open-weights guards across two contract corpora, document-level false positive rates reached 92.39%; two guards scored below chance on paired AUC, and a five-regex baseline was not Pareto-dominated by any guard they tested. The authors say controlled probes exposed both model-specific and general failure modes. Prompted LLMs resolved the tested failures, including unauthorized scope claims, a threat category on which every guard scored zero. They also report that a cascaded LLM solution—using a mid-tier model and having a frontier model verify its positive results—cost 6.5x less than using only a frontier model while approaching its performance. Escape Meridian interpretation: the reported false positives make visible a practical tension: guards intended to protect document analysis systems may themselves flag contract content, while the proposed cascade adds a verification step and reports lower cost. The abstract reports these results but does not provide the evaluation details needed to assess how the findings might carry over to other contracts or systems.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the paper's abstract; it does not include the evaluation methods, corpus details, or results needed to independently assess the reported findings. The abstract reports results for two corpora of legal contracts and the evaluated guards; it does not establish how those results generalize to other contracts or systems.","speaker":{"name":"Heimann, Mark, Taylor, Katherine, Jacob, Bruno","roleAtSource":"Article author"},"source":{"title":"Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.08834","publishedAt":"2026-11-06","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors evaluated eleven open-weights guards on two corpora of legal contracts.","excerpt":"We measure eleven open-weights guards on two corpora of legal contracts","url":"https://arxiv.org/abs/2611.08834","relationship":"Source-stated"},{"text":"The authors report document-level false positive rates as high as 92.39%, and two guards fell below chance on paired AUC.","excerpt":"document-level false positive rates are as high as 92.39%; two guards fall below chance on paired AUC","url":"https://arxiv.org/abs/2611.08834","relationship":"Source-stated"},{"text":"A five-regex baseline was not Pareto-dominated by any guard in the panel.","excerpt":"a five-regex baseline is not Pareto-dominated by any guard in the panel.","url":"https://arxiv.org/abs/2611.08834","relationship":"Source-stated"},{"text":"The authors say prompted LLMs resolved the tested failures, including unauthorized scope claims, where every guard scored zero.","excerpt":"including a second threat category (unauthorized scope claims) on which every guard scores zero.","url":"https://arxiv.org/abs/2611.08834","relationship":"Source-stated"},{"text":"The authors report that a cascade using a mid-tier model and a frontier model to verify its positive results costs 6.5x less than using only a frontier model, while approaching its performance.","excerpt":"A cascaded LLM solution, where the positives of a mid-tier model are verified by a frontier model, is 6.5x cheaper than using exclusively a frontier model and approaches its performance.","url":"https://arxiv.org/abs/2611.08834","relationship":"Source-stated"},{"text":"The abstract does not provide enough evaluation detail to assess how the findings carry over to other contracts or systems.","excerpt":"We measure eleven open-weights guards on two corpora of legal contracts","url":"https://arxiv.org/abs/2611.08834","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.08834","checkedAt":"2026-11-09T07:50:08.143Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What evaluation details would help determine whether the reported guard and cascade results carry over beyond the two contract corpora and systems tested?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Measuring Prompt Injection Guards on Legal Contract Review Attacks","summary":"Mark Heimann, Katherine Taylor and Bruno Jacob report that specialized prompt injection guards tested on legal contracts can miss attacks and misclassify ordinary document content. In their evaluation of eleven open-weights guards across two contract corpora, document-level false positive rates reached 92.39%; two guards scored below chance on paired AUC, and a five-regex baseline was not Pareto-dominated by any guard they tested. The authors say controlled probes exposed both model-specific and general failure modes. Prompted LLMs resolved the tested failures, including unauthorized scope claims, a threat category on which every guard scored zero. They also report that a cascaded LLM solution—using a mid-tier model and having a frontier model verify its positive results—cost 6.5x less than using only a frontier model while approaching its performance. Escape Meridian interpretation: the reported false positives make visible a practical tension: guards intended to protect document analysis systems may themselves flag contract content, while the proposed cascade adds a verification step and reports lower cost. The abstract reports these results but does not provide the evaluation details needed to assess how the findings might carry over to other contracts or systems.","question":"What evaluation details would help determine whether the reported guard and cascade results carry over beyond the two contract corpora and systems tested?","claims":[{"text":"The authors evaluated eleven open-weights guards on two corpora of legal contracts.","excerpt":"We measure eleven open-weights guards on two corpora of legal contracts","support":"source-stated"},{"text":"The authors report document-level false positive rates as high as 92.39%, and two guards fell below chance on paired AUC.","excerpt":"document-level false positive rates are as high as 92.39%; two guards fall below chance on paired AUC","support":"source-stated"},{"text":"A five-regex baseline was not Pareto-dominated by any guard in the panel.","excerpt":"a five-regex baseline is not Pareto-dominated by any guard in the panel.","support":"source-stated"},{"text":"The authors say prompted LLMs resolved the tested failures, including unauthorized scope claims, where every guard scored zero.","excerpt":"including a second threat category (unauthorized scope claims) on which every guard scores zero.","support":"source-stated"},{"text":"The authors report that a cascade using a mid-tier model and a frontier model to verify its positive results costs 6.5x less than using only a frontier model, while approaching its performance.","excerpt":"A cascaded LLM solution, where the positives of a mid-tier model are verified by a frontier model, is 6.5x cheaper than using exclusively a frontier model and approaches its performance.","support":"source-stated"},{"text":"The abstract does not provide enough evaluation detail to assess how the findings carry over to other contracts or systems.","excerpt":"We measure eleven open-weights guards on two corpora of legal contracts","support":"interpretation"}],"limitations":["The supplied evidence is the paper's abstract; it does not include the evaluation methods, corpus details, or results needed to independently assess the reported findings.","The abstract reports results for two corpora of legal contracts and the evaluated guards; it does not establish how those results generalize to other contracts or systems."],"mediaId":"0fb5fb29201a559a9abfcdee1eebffb647b2fb07faa4d12ceb2bb2b86c5903b2","correctionReason":null,"id":"8d6a3f250cbef27e45979f9a797fb394df1be103230adfed6d2e320b381475ef","publicationId":"0bba5917-0aa2-49b4-832b-e6a04df62bf6","version":1,"sourceUrl":"https://arxiv.org/abs/2611.08834","sourceTitle":"Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-06","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-09T05:03:46.697Z","createdAt":"2026-11-09T06:46:33.176Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"0fb5fb29201a559a9abfcdee1eebffb647b2fb07faa4d12ceb2bb2b86c5903b2","kind":"article","url":"https://arxiv.org/abs/2611.08834","sourceUrl":"https://arxiv.org/abs/2611.08834","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T06:16:46.410Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.08834","checkedAt":"2026-11-09T07:50:08.143Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"0fb5fb29201a559a9abfcdee1eebffb647b2fb07faa4d12ceb2bb2b86c5903b2","confidence":0.92,"provider":"typesafe","model":"jev-1.13.0","inputHash":"772a66c688a770c48d91b468b7047c9ff9c26eceb992a77cf98b5bfe3db38c79","decidedAt":"2026-11-09T06:16:48.309Z"},"provenance":{"textHash":"9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d","observationId":"52e84440789f91d5edaecb2684275c079244da30fd6072eaa4675c136256f0a2","retrievedAt":"2026-11-09T06:16:46.125Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"21261a93-c3f2-4396-88f5-00aeed020600"},"reviewer":{"model":"gpt-6-luna","runId":"227386ae-94aa-406a-89fc-59adae26c837"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T06:46:33.176Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Heimann, Mark","url":null,"kind":"unknown"},{"name":"Taylor, Katherine","url":null,"kind":"unknown"},{"name":"Jacob, Bruno","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Heimann, Mark","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Heimann, Mark","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08834","locator":"head/meta[name=author|citation_author]"},"assertionId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:50:08.143Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2","sourceUrl":"https://arxiv.org/abs/2611.08834","contentSha256":"244679eb1a6233c22f1d2aaef73ca77c43fc70423e5a1bde8e846f59607e0a78","textHash":"9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d","observedAt":"2026-11-09T07:50:08.143Z"}},{"actor":{"name":"Taylor, Katherine","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Taylor, Katherine","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08834","locator":"head/meta[name=author|citation_author]"},"assertionId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:50:08.143Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2","sourceUrl":"https://arxiv.org/abs/2611.08834","contentSha256":"244679eb1a6233c22f1d2aaef73ca77c43fc70423e5a1bde8e846f59607e0a78","textHash":"9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d","observedAt":"2026-11-09T07:50:08.143Z"}},{"actor":{"name":"Jacob, Bruno","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jacob, Bruno","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08834","locator":"head/meta[name=author|citation_author]"},"assertionId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:50:08.143Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2","sourceUrl":"https://arxiv.org/abs/2611.08834","contentSha256":"244679eb1a6233c22f1d2aaef73ca77c43fc70423e5a1bde8e846f59607e0a78","textHash":"9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d","observedAt":"2026-11-09T07:50:08.143Z"}}],"participationCoverage":{"total":3,"returned":3,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.08834","snapshotId":"bdb79043ed946a034c6ba61832025fbfed0978c72e779a2a73b84d46f4df2cb2","contentSha256":"244679eb1a6233c22f1d2aaef73ca77c43fc70423e5a1bde8e846f59607e0a78","textHash":"9f182229a7b8895a5ea31050ba93923fb19c7d1ff742e175f31cbeffaf06a44d","observedAt":"2026-11-09T07:50:08.143Z","basis":"publisher-page-metadata","sourceLinks":41,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/8d6a3f250cbef27e45979f9a797fb394df1be103230adfed6d2e320b381475ef/source"}}}],"audits":[{"publicationId":"0bba5917-0aa2-49b4-832b-e6a04df62bf6","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The draft correctly names authors Mark Heimann, Katherine Taylor and Bruno Jacob. It frames the findings as their reported results, and labels the closing inference as Escape Meridian interpretation. 'Prompted LLMs resolved the tested failures' is not explicitly attributed, but it follows 'The authors say' and stays within the abstract's reporting frame.","excerpt":"Authors:Mark Heimann, Katherine Taylor, Bruno Jacob"},{"dimension":"summary","status":"uncertain","explanation":"The numbers and findings are faithful, but central terms go unexplained for a non-specialist: prompt injection guard, open-weights, paired AUC, Pareto-dominated, regex and frontier model. The paper's notable point is that a simple five-regex baseline is not beaten on both error trade-offs by any guard. That point is hidden behind 'Pareto-dominated'. Smallest repair: one plain sentence saying a prompt injection guard screens documents for hidden instructions aimed at an AI. Add another saying no guard beat a simple five-pattern text filter on both catching attacks and avoiding false alarms. Tooltips could also cover these terms.","excerpt":"a five-regex baseline is not Pareto-dominated by any guard in the panel"},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-11-06. That matches the arXiv v1 submission date, not the 2026-11-09 feed date.","excerpt":"[Submitted on 6 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"All reported figures match the abstract: eleven guards, two corpora, 92.39% FPR, below-chance paired AUC, the regex baseline, zero scores on unauthorized scope claims, and the 6.5x cheaper cascade that approaches frontier performance. 'Can miss attacks' is supported by the every-guard-zero result. The labeled interpretation about guards flagging contract content follows from the FPR finding. The limitation and question stay conditional and do not claim real-world reliability.","excerpt":"Prompted LLMs resolve the tested failures, including a second threat category (unauthorized scope claims) on which every guard scores zero."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper. It is the most direct access to the source.","excerpt":null}],"createdAt":"2026-11-09T06:46:33.176Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}