Huang, Yini, LU, Aiwen, Sun, Zhen, Han, Yi, Zhao, Yuzhi, Wei, Jiaheng · Oct 30, 2026
arXiv cs.AI: ANGLER: Verifier-Grounded Benchmarking with an Adaptive Attacker for Web Agents
Markdown · JSON · Original source
The authors of ANGLER introduce REEF, a for testing web agents against adversarial content that could induce unauthorized actions, and ANGLER, a framework that adapts attacks using outcomes from earlier attempts. They argue that fixed attacks and evaluations focused on text can miss vulnerabilities revealed by real website state changes. REEF pairs benign tasks with malicious objectives across websites and attack types. Its state-based verifiers provide feedback that ANGLER uses to refine attacks, prioritizing near-miss seeds. In tests across VLM-based web agents, the authors report a higher end-to-end attack success rate with ANGLER than in the non-adaptive setting. They also report that these attacks reduce benign task success compared with attack-free execution on the same tasks. The authors’ results show how attacks can both pursue a malicious objective and disrupt the user’s benign task in this benchmark. The abstract does not establish how well these results generalize beyond the evaluated settings. How do ANGLER’s results vary across agents, websites, and tasks beyond those evaluated?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 30, 2026 (2026-10-30).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-11T01:25:29.885Z. First observed: 2026-11-02T05:03:46.163Z. Retrieved: 2026-11-03T05:19:24.067Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The evidence is the paper's abstract; it does not provide details of the evaluation methods or results beyond the reported figures. The abstract does not establish whether the reported results generalize beyond the evaluated websites, tasks, and agents.
Open question
How do ANGLER’s results vary across agents, websites, and tasks beyond those evaluated?
Source-stated
The authors introduce REEF as a benchmark for measuring web-agent security vulnerabilities in executable environments with verification.
we introduce REEF, a benchmark of red-teaming executable environments with faithful verification for measuring the security vulnerabilities of web agents.
Source-stated
REEF includes verifier-grounded websites and pairs benign tasks with malicious objectives delivered through multiple attack types.
REEF covers 9 verifier-grounded websites spanning travel, commerce, productivity, media, and communication, and pairs every authentic benign task with a malicious objective delivered through 5 attack types at both the user and the environment level.
Source-stated
ANGLER uses outcomes from previous rollouts to guide attack mutation and selection, prioritizing near-miss seeds.
ANGLER uses outcomes from prior rollouts to guide LLM-based attack mutation and candidate selection, prioritizing near-miss seeds during refinement.
Source-stated
Across the evaluated VLM-based web agents, the authors report a higher end-to-end attack success rate for ANGLER than for the non-adaptive setting.
Across 8 VLM-based web agents, ANGLER achieves an end-to-end attack success rate (ASR) of 20.36%, compared with 15.24% for the non-adaptive setting.
Source-stated
The authors report that attacks generated by ANGLER reduce benign task success compared with attack-free execution on the same tasks.
Compared with attack-free execution on the same benign tasks, attacks generated by ANGLER reduce benign task success rate (BSR) from 61.9% to 44.4%.
Source and contributor context
Huang, Yini, LU, Aiwen, Sun, Zhen, Han, Yi, Zhao, Yuzhi, Wei, Jiaheng · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: ea0d81f1-df1e-40da-8201-0b1faa4a539f.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-11T01:25:29.885Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 87efcce4226b9d8b1a6b6d154312262f75f4e43303a33642a87ff49c4239e97b. Source text hash: 9c5fc33ea9232e5428542e7aad786c58723b2e7d382e9ebcd466ab85bc1ec91b.
Revision and source-check history
Version 1 · 2026-11-11T01:25:29.885Z
arXiv cs.AI: ANGLER: Verifier-Grounded Benchmarking with an Adaptive Attacker for Web Agents
The authors of ANGLER introduce REEF, a for testing web agents against adversarial content that could induce unauthorized actions, and ANGLER, a framework that adapts attacks using outcomes from earlier attempts. They argue that fixed attacks and evaluations focused on text can miss vulnerabilities revealed by real website state changes. REEF pairs benign tasks with malicious objectives across websites and attack types. Its state-based verifiers provide feedback that ANGLER uses to refine attacks, prioritizing near-miss seeds. In tests across VLM-based web agents, the authors report a higher end-to-end attack success rate with ANGLER than in the non-adaptive setting. They also report that these attacks reduce benign task success compared with attack-free execution on the same tasks. The authors’ results show how attacks can both pursue a malicious objective and disrupt the user’s benign task in this benchmark. The abstract does not establish how well these results generalize beyond the evaluated settings. How do ANGLER’s results vary across agents, websites, and tasks beyond those evaluated?
Original source
cs.AI · Article
Source details
Original source date: Oct 30, 2026 (2026-10-30).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-11T01:25:29.885Z. First observed: 2026-11-02T05:03:46.163Z. Retrieved: 2026-11-03T05:19:24.067Z.
Limits
The evidence is the paper's abstract; it does not provide details of the evaluation methods or results beyond the reported figures. The abstract does not establish whether the reported results generalize beyond the evaluated websites, tasks, and agents.
Open question
How do ANGLER’s results vary across agents, websites, and tasks beyond those evaluated?
Source-stated
The authors introduce REEF as a benchmark for measuring web-agent security vulnerabilities in executable environments with verification.
we introduce REEF, a benchmark of red-teaming executable environments with faithful verification for measuring the security vulnerabilities of web agents.
Source-stated
REEF includes verifier-grounded websites and pairs benign tasks with malicious objectives delivered through multiple attack types.
REEF covers 9 verifier-grounded websites spanning travel, commerce, productivity, media, and communication, and pairs every authentic benign task with a malicious objective delivered through 5 attack types at both the user and the environment level.
Source-stated
ANGLER uses outcomes from previous rollouts to guide attack mutation and selection, prioritizing near-miss seeds.
ANGLER uses outcomes from prior rollouts to guide LLM-based attack mutation and candidate selection, prioritizing near-miss seeds during refinement.
Source-stated
Across the evaluated VLM-based web agents, the authors report a higher end-to-end attack success rate for ANGLER than for the non-adaptive setting.
Across 8 VLM-based web agents, ANGLER achieves an end-to-end attack success rate (ASR) of 20.36%, compared with 15.24% for the non-adaptive setting.
Source-stated
The authors report that attacks generated by ANGLER reduce benign task success compared with attack-free execution on the same tasks.
Compared with attack-free execution on the same benign tasks, attacks generated by ANGLER reduce benign task success rate (BSR) from 61.9% to 44.4%.
Source and contributor context
Huang, Yini, LU, Aiwen, Sun, Zhen, Han, Yi, Zhao, Yuzhi, Wei, Jiaheng · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: ea0d81f1-df1e-40da-8201-0b1faa4a539f.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-11T01:25:29.885Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 87efcce4226b9d8b1a6b6d154312262f75f4e43303a33642a87ff49c4239e97b. Source text hash: 9c5fc33ea9232e5428542e7aad786c58723b2e7d382e9ebcd466ab85bc1ec91b.
uncertain · 2026-11-11T01:25:29.885Z · anthropic/claude-opus-5.5
Publication: ea0d81f1-df1e-40da-8201-0b1faa4a539f
attribution — pass: The reading attributes REEF, ANGLER and all results to the paper's authors (Yini Huang et al.) and keeps the findings as reported claims. The title prefix 'arXiv cs.AI' matches the feed publisher. The paper's primary category is cs.CR and it is cross-listed in cs.AI, which does not misattribute the work.
Authors:Yini Huang, Aiwen LU, Zhen Sun, Yi Han, Yuzhi Zhao, Jiaheng Wei
summary — uncertain: The reading is faithful but hard for a nonspecialist to follow. Central terms are left unexplained: web agents (AI systems that act on websites for users), VLM-based, near-miss seeds, state-based verifiers and red-teaming. It also drops the scale the source reports: 9 websites, 8 agents, 20.36% vs 15.24% attack success rate, and benign success falling from 61.9% to 44.4%. Saying only 'higher' and 'reduce' leaves readers unable to judge the size of the effect. Smallest repair: restore the figures, add one plain-language gloss for web agents and the adaptive near-miss idea, and add term tooltips for VLM and ASR/BSR.
ANGLER achieves an end-to-end attack success rate (ASR) of 20.36%, compared with 15.24% for the non-adaptive setting.
date — pass: The originalPublished date of 2026-10-30 matches the v1 submission date on the arXiv page. The feed date of 2026-11-02 is later and correctly not used.
[Submitted on 30 Oct 2026]
support — pass: Each claim matches its abstract excerpt. The characterization of the paper's argument about fixed and text-level evaluations reflects the source's own 'underestimating' framing. The limitations and the question about generalization are stated as unknowns rather than invented conclusions, and no benefits or costs are asserted beyond the source.
evaluate attacks at the text level rather than through real state changes, thus underestimating how vulnerable agents are to diverse attacks
media — pass: The selected media is the verified direct link to the arXiv abstract page, which is the original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv