# arXiv cs.AI: Beyond Instruction Override: Benchmarking Indirect Prompt Injection with Task-Tampering Attacks [Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-beyond-instruction-override-benchmarking-indirect-123627db) Rongting Zhang, Martin Bertran, Sandeep Avula, Chia-Jung Lee and Vanessa Murdock argue that indirect prompt injection can tamper with a task’s decisions without replacing the task. Their benchmark treats planted content as an attack when its source lacks authority over the decision it tries to influence. For example, dispute evidence might cite a false refund policy to inflate a refund, while the agent still works on the buyer’s dispute. Across 90 validated cases in three tool-use environments and five LLMs, the authors report that attacks on decision rules, available actions and state had higher mean success than attacks on the agent’s objective. A detection-and-removal defense reduced objective-targeting attacks to 2.4% success, but observation- and state-targeting attacks succeeded at 9.5% and 12.1%. A tool-policy defense allowed 15.4% attack success, close to 17.0% without a defense; the authors say these attacks generally keep the expected tools and manipulate values gathered during execution. The authors argue that evaluations focused on redirecting tasks or targeting objectives can overestimate how much these defenses cover. The benchmark makes the source’s authority over a decision central to defining the attack, and its results suggest that defenses tested against task replacement may miss attacks that leave the task intact. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 5, 2026 (2026-11-05). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-08T20:20:17.870Z. First observed: 2026-11-06T05:12:39.172Z. Retrieved: 2026-11-08T20:19:09.778Z. - [Beyond Instruction Override: Benchmarking Indirect Prompt Injection with Task-Tampering Attacks](https://arxiv.org/abs/2611.07640) ## Limits The supplied evidence is the paper’s abstract; it does not name the five LLMs or provide further detail about the benchmark cases or evaluation procedures. ## Open question How well do these attack patterns and defense results hold beyond the 90 validated cases, three tool-use environments and five LLMs? ## Source-stated The authors define an indirect prompt-injection attack by whether planted content tries to influence a decision its source is not authorized to determine, rather than by whether it uses instruction-like wording. > planted content constitutes an attack when it attempts to influence a decision that its source is not authorized to determine - [Supporting source](https://arxiv.org/abs/2611.07640) ## Source-stated The benchmark contains 90 validated cases across three tool-use environments, each targeting a decision required by the benign task rather than replacing the task. > in each of 90 validated cases across three tool-use environments - [Supporting source](https://arxiv.org/abs/2611.07640) ## Source-stated Across five LLMs, attacks targeting decision rules, available actions and state had higher mean success than objective-targeting attacks without a defense. > attacks on three of the five other targets (decision rules, available actions, and state) have higher mean success than objective attacks across five LLMs - [Supporting source](https://arxiv.org/abs/2611.07640) ## Source-stated The detection-and-removal defense reduced objective-targeting attacks to 2.4% success but left observation- and state-targeting attacks at 9.5% and 12.1%. > reduces objective-targeting attacks to 2.4% success but leaves observation- and state-targeting attacks at 9.5% and 12.1% - [Supporting source](https://arxiv.org/abs/2611.07640) ## Source-stated The tool-policy defense allowed 15.4% attack success, compared with 17.0% without a defense; the authors say these attacks generally retain expected tools and manipulate values obtained during execution. > A tool-policy defense still allows 15.4% attack success, close to 17.0% without a defense - [Supporting source](https://arxiv.org/abs/2611.07640) ## Source-stated The authors caution that evaluations focused on task redirection or objective-targeting attacks can overestimate defense coverage. > Evaluations whose attacks redirect the task or target only the agent's objective can therefore overestimate the coverage of these defenses. - [Supporting source](https://arxiv.org/abs/2611.07640) ## Source and contributor context Zhang, Rongting, Bertran, Martin, Avula, Sandeep, Lee, Chia-Jung, Murdock, Vanessa · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## Prompt injection Hidden instructions placed in a document, email or web page that try to take control of an AI system reading it. - [Definition reference](https://en.wikipedia.org/wiki/Prompt_injection) ## Tool use A model's ability to operate other software, such as a search engine, a calculator or a code runner, while it works. - [Definition reference](https://docs.anthropic.com/en/docs/build-with-claude/tool-use/overview) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 3005b244-cb90-4134-8fa6-09ca1b0734b4. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-08T20:20:17.870Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: eea00d7603ae76c783f25f89daf6eeb5a61d1b1a6f604392b4614a8cea9c0ed3. Source text hash: 749292f1d40fde1b183ae869e17cd81e63ecc072f03196242f62069bde51b324. ## Revision and source-check history ### Version 1 — 2026-11-08T20:20:17.870Z Rongting Zhang, Martin Bertran, Sandeep Avula, Chia-Jung Lee and Vanessa Murdock argue that indirect prompt injection can tamper with a task’s decisions without replacing the task. Their benchmark treats planted content as an attack when its source lacks authority over the decision it tries to influence. For example, dispute evidence might cite a false refund policy to inflate a refund, while the agent still works on the buyer’s dispute. Across 90 validated cases in three tool-use environments and five LLMs, the authors report that attacks on decision rules, available actions and state had higher mean success than attacks on the agent’s objective. A detection-and-removal defense reduced objective-targeting attacks to 2.4% success, but observation- and state-targeting attacks succeeded at 9.5% and 12.1%. A tool-policy defense allowed 15.4% attack success, close to 17.0% without a defense; the authors say these attacks generally keep the expected tools and manipulate values gathered during execution. The authors argue that evaluations focused on redirecting tasks or targeting objectives can overestimate how much these defenses cover. The benchmark makes the source’s authority over a decision central to defining the attack, and its results suggest that defenses tested against task replacement may miss attacks that leave the task intact. ## Original source arXiv cs.AI · Article Original source date: Nov 5, 2026 (2026-11-05). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-08T20:20:17.870Z. First observed: 2026-11-06T05:12:39.172Z. Retrieved: 2026-11-08T20:19:09.778Z. - [Beyond Instruction Override: Benchmarking Indirect Prompt Injection with Task-Tampering Attacks](https://arxiv.org/abs/2611.07640) ## Limits The supplied evidence is the paper’s abstract; it does not name the five LLMs or provide further detail about the benchmark cases or evaluation procedures. ## Open question How well do these attack patterns and defense results hold beyond the 90 validated cases, three tool-use environments and five LLMs? ## Source-stated The authors define an indirect prompt-injection attack by whether planted content tries to influence a decision its source is not authorized to determine, rather than by whether it uses instruction-like wording. > planted content constitutes an attack when it attempts to influence a decision that its source is not authorized to determine - [Supporting source](https://arxiv.org/abs/2611.07640) ## Source-stated The benchmark contains 90 validated cases across three tool-use environments, each targeting a decision required by the benign task rather than replacing the task. > in each of 90 validated cases across three tool-use environments - [Supporting source](https://arxiv.org/abs/2611.07640) ## Source-stated Across five LLMs, attacks targeting decision rules, available actions and state had higher mean success than objective-targeting attacks without a defense. > attacks on three of the five other targets (decision rules, available actions, and state) have higher mean success than objective attacks across five LLMs - [Supporting source](https://arxiv.org/abs/2611.07640) ## Source-stated The detection-and-removal defense reduced objective-targeting attacks to 2.4% success but left observation- and state-targeting attacks at 9.5% and 12.1%. > reduces objective-targeting attacks to 2.4% success but leaves observation- and state-targeting attacks at 9.5% and 12.1% - [Supporting source](https://arxiv.org/abs/2611.07640) ## Source-stated The tool-policy defense allowed 15.4% attack success, compared with 17.0% without a defense; the authors say these attacks generally retain expected tools and manipulate values obtained during execution. > A tool-policy defense still allows 15.4% attack success, close to 17.0% without a defense - [Supporting source](https://arxiv.org/abs/2611.07640) ## Source-stated The authors caution that evaluations focused on task redirection or objective-targeting attacks can overestimate defense coverage. > Evaluations whose attacks redirect the task or target only the agent's objective can therefore overestimate the coverage of these defenses. - [Supporting source](https://arxiv.org/abs/2611.07640) ## Source and contributor context Zhang, Rongting, Bertran, Martin, Avula, Sandeep, Lee, Chia-Jung, Murdock, Vanessa · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## Prompt injection Hidden instructions placed in a document, email or web page that try to take control of an AI system reading it. - [Definition reference](https://en.wikipedia.org/wiki/Prompt_injection) ## Tool use A model's ability to operate other software, such as a search engine, a calculator or a code runner, while it works. - [Definition reference](https://docs.anthropic.com/en/docs/build-with-claude/tool-use/overview) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 3005b244-cb90-4134-8fa6-09ca1b0734b4. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-08T20:20:17.870Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: eea00d7603ae76c783f25f89daf6eeb5a61d1b1a6f604392b4614a8cea9c0ed3. Source text hash: 749292f1d40fde1b183ae869e17cd81e63ecc072f03196242f62069bde51b324. ### uncertain — 2026-11-08T20:20:17.870Z — anthropic/claude-opus-5.5 Publication: 3005b244-cb90-4134-8fa6-09ca1b0734b4 attribution — pass: The draft correctly names the five arXiv authors and attributes the definition, benchmark results and coverage caution to them. The 'arXiv cs.AI' label matches the feed publisher; the paper is primary cs.CR and cross-listed in cs.AI. > Authors:Rongting Zhang, Martin Bertran, Sandeep Avula, Chia-Jung Lee, Vanessa Murdock summary — uncertain: The draft is faithful and keeps the authors' central argument. However, the opening sentence uses the central term 'indirect prompt injection' without a plain-language explanation. It also refers to 'the agent' without saying this means an AI system that uses tools and reads outside content. The refund example helps, but a non-specialist may still miss the basic idea. Smallest repair: add a short gloss or tooltip, such as: content an AI agent reads while doing a task (for example, a buyer's dispute evidence) that tries to steer its decisions. The question is grounded in the evidence. > Indirect prompt-injection (IPI) benchmarks predominantly test planted instructions that redirect a tool-using agent toward an attacker-chosen task. date — pass: The originalPublished date of 2026-11-05 matches the v1 submission date. The feed date of Nov 6 is later and is correctly not used. > \[Submitted on 5 Nov 2026\] support — pass: All the figures (90 cases, three environments, five LLMs, 2.4%, 9.5%, 12.1%, 15.4%, 17.0%) and the comparison between attack targets match the abstract. The closing inference that defenses tested against task replacement may miss tampering attacks restates the authors' own conclusion and is not overstated. The draft omits that the gap persists under detection-optimized rewrites, but this omission does not distort the meaning. > Evaluations whose attacks redirect the task or target only the agent's objective can therefore overestimate the coverage of these defenses. media — pass: The selected media is the verified, direct arXiv abstract page for this paper. It is the best available link to the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/123627dbd658290e63b4583460666027b83cc682542c700825688d785343840a/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-beyond-instruction-override-benchmarking-indirect-123627db.json)