# arXiv paper introduces ScamWorld to test web agents against online fraud [Canonical reading](https://escapemeridian.com/readings/arxiv-paper-introduces-scamworld-to-test-web-agents-against-adf217e2) Authors Yutao Wu, Xiao Liu, Ming Wen, Hanxun Huang, Jun Wang and Xingjun Ma introduce ScamWorld, an interactive environment for testing how web agents handle online fraud. They argue that this is an underexplored problem: agents encounter fraudulent stores, fake investment platforms and advance-fee schemes, while scam websites are often short-lived and difficult to evaluate systematically. ScamWorld reconstructs reported scam websites in different scenarios, with documented fraud mechanisms made observable before an agent’s final action. The authors say this enables controlled, repeatable evaluation. In tests of agents powered by six frontier LLMs, baseline deception rates ranged from 11% to 35% across models. Agents proceeded more readily with payments than with requests for credentials or wallet information. The authors also report that some agents acknowledged suspicious conditions but proceeded anyway, sometimes interpreting a user’s instructions as permission to accept additional terms imposed by a website. The environment is presented as a reusable testbed for evaluating and stress-testing web agents against online fraud. Escape Meridian’s interpretation: the abstract reports these results but does not identify the six models, so the rates cannot be assigned to particular models from this evidence. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 3, 2026 (2026-11-03). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-10T04:02:55.846Z. First observed: 2026-11-04T05:18:11.754Z. Retrieved: 2026-11-10T04:02:17.076Z. - [ScamWorld: An Interactive Environment for Evaluating Web Agents Against Online Fraud](https://arxiv.org/abs/2611.04879) ## Limits The evidence is the paper's abstract and does not give the names of the six tested models or model-specific deception rates. The abstract describes the environment and findings but does not provide detailed evaluation procedures or results beyond the reported comparisons. ## Open question How do the reported deception rates and agent behaviours vary across the six frontier LLMs tested? ## Source-stated The authors introduce ScamWorld as an interactive environment that reconstructs reported scam websites for evaluating web agents against online fraud. > we introduce ScamWorld, an interactive environment that reconstructs reported scam websites across diverse scenarios, fraud mechanisms, and languages. - [Supporting source](https://arxiv.org/abs/2611.04879) ## Source-stated ScamWorld makes documented fraud mechanisms observable before an agent's final action, enabling controlled, repeatable evaluation. > Each reconstructed website reproduces documented fraud mechanisms, making relevant signs of fraud observable before the agent's final action and enabling controlled, repeatable evaluation. - [Supporting source](https://arxiv.org/abs/2611.04879) ## Source-stated Across agents powered by six frontier LLMs, the authors report baseline deception rates from 11% to 35%. > Through an extensive study of web agents powered by six frontier LLMs, we find that baseline deception rates range from 11% to 35% across models. - [Supporting source](https://arxiv.org/abs/2611.04879) ## Source-stated The authors report that agents proceed more readily with payments than with requests for credentials or wallet information. > Agents proceed more readily with payments than with requests for credentials or wallet information. - [Supporting source](https://arxiv.org/abs/2611.04879) ## Source-stated Some agents reportedly recognize suspicious conditions and proceed anyway, sometimes treating user instructions as authorization to accept extra terms imposed by a website. > some agents acknowledge suspicious conditions yet proceed, at times treating the user's instructions as authorisation to accept additional terms imposed by the website. - [Supporting source](https://arxiv.org/abs/2611.04879) ## Escape Meridian interpretation The abstract does not name the six tested models, so its reported rates cannot be attributed to particular models using this evidence. > baseline deception rates range from 11% to 35% across models. - [Supporting source](https://arxiv.org/abs/2611.04879) ## Source and contributor context Wu, Yutao, Liu, Xiao, Wen, Ming, Huang, Hanxun, Wang, Jun, Ma, Xingjun · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 15690c40-3e8d-43b8-8945-316c9af7c0c8. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T04:02:55.846Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 489fdade7c2012f0b3c3bcb3314941884b5fa82ca17e0c25e9926238288eab15. Source text hash: 24d48bfde8fadf7fb1643174f11456f0603b28f155d8935da0b17eb6432a7f38. ## Revision and source-check history ### Version 1 — 2026-11-10T04:02:55.846Z Authors Yutao Wu, Xiao Liu, Ming Wen, Hanxun Huang, Jun Wang and Xingjun Ma introduce ScamWorld, an interactive environment for testing how web agents handle online fraud. They argue that this is an underexplored problem: agents encounter fraudulent stores, fake investment platforms and advance-fee schemes, while scam websites are often short-lived and difficult to evaluate systematically. ScamWorld reconstructs reported scam websites in different scenarios, with documented fraud mechanisms made observable before an agent’s final action. The authors say this enables controlled, repeatable evaluation. In tests of agents powered by six frontier LLMs, baseline deception rates ranged from 11% to 35% across models. Agents proceeded more readily with payments than with requests for credentials or wallet information. The authors also report that some agents acknowledged suspicious conditions but proceeded anyway, sometimes interpreting a user’s instructions as permission to accept additional terms imposed by a website. The environment is presented as a reusable testbed for evaluating and stress-testing web agents against online fraud. Escape Meridian’s interpretation: the abstract reports these results but does not identify the six models, so the rates cannot be assigned to particular models from this evidence. ## Original source arXiv cs.AI · Article Original source date: Nov 3, 2026 (2026-11-03). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-10T04:02:55.846Z. First observed: 2026-11-04T05:18:11.754Z. Retrieved: 2026-11-10T04:02:17.076Z. - [ScamWorld: An Interactive Environment for Evaluating Web Agents Against Online Fraud](https://arxiv.org/abs/2611.04879) ## Limits The evidence is the paper's abstract and does not give the names of the six tested models or model-specific deception rates. The abstract describes the environment and findings but does not provide detailed evaluation procedures or results beyond the reported comparisons. ## Open question How do the reported deception rates and agent behaviours vary across the six frontier LLMs tested? ## Source-stated The authors introduce ScamWorld as an interactive environment that reconstructs reported scam websites for evaluating web agents against online fraud. > we introduce ScamWorld, an interactive environment that reconstructs reported scam websites across diverse scenarios, fraud mechanisms, and languages. - [Supporting source](https://arxiv.org/abs/2611.04879) ## Source-stated ScamWorld makes documented fraud mechanisms observable before an agent's final action, enabling controlled, repeatable evaluation. > Each reconstructed website reproduces documented fraud mechanisms, making relevant signs of fraud observable before the agent's final action and enabling controlled, repeatable evaluation. - [Supporting source](https://arxiv.org/abs/2611.04879) ## Source-stated Across agents powered by six frontier LLMs, the authors report baseline deception rates from 11% to 35%. > Through an extensive study of web agents powered by six frontier LLMs, we find that baseline deception rates range from 11% to 35% across models. - [Supporting source](https://arxiv.org/abs/2611.04879) ## Source-stated The authors report that agents proceed more readily with payments than with requests for credentials or wallet information. > Agents proceed more readily with payments than with requests for credentials or wallet information. - [Supporting source](https://arxiv.org/abs/2611.04879) ## Source-stated Some agents reportedly recognize suspicious conditions and proceed anyway, sometimes treating user instructions as authorization to accept extra terms imposed by a website. > some agents acknowledge suspicious conditions yet proceed, at times treating the user's instructions as authorisation to accept additional terms imposed by the website. - [Supporting source](https://arxiv.org/abs/2611.04879) ## Escape Meridian interpretation The abstract does not name the six tested models, so its reported rates cannot be attributed to particular models using this evidence. > baseline deception rates range from 11% to 35% across models. - [Supporting source](https://arxiv.org/abs/2611.04879) ## Source and contributor context Wu, Yutao, Liu, Xiao, Wen, Ming, Huang, Hanxun, Wang, Jun, Ma, Xingjun · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 15690c40-3e8d-43b8-8945-316c9af7c0c8. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T04:02:55.846Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 489fdade7c2012f0b3c3bcb3314941884b5fa82ca17e0c25e9926238288eab15. Source text hash: 24d48bfde8fadf7fb1643174f11456f0603b28f155d8935da0b17eb6432a7f38. ### uncertain — 2026-11-10T04:02:55.846Z — anthropic/claude-opus-5.5 Publication: 15690c40-3e8d-43b8-8945-316c9af7c0c8 attribution — pass: The draft correctly names the six listed authors and attributes the environment, findings and the 'underexplored' framing to them. Results are hedged as reported by the authors, and the closing point is labeled as Escape Meridian interpretation. > Authors:Yutao Wu, Xiao Liu, Ming Wen, Hanxun Huang, Jun Wang, Xingjun Ma summary — uncertain: The summary is faithful, but its central term is never explained. It does not say what a web agent is or why fraud matters for one. The abstract supplies this: agents browse, shop and manage accounts for users, so being deceived could mean paying or handing over credentials on a user's behalf. Without it, a non-specialist misses the stake, and 'baseline deception rate' has no context. Smallest repair: in the first sentence, add the source's description that web agents browse, shop and manage accounts on behalf of users. It could also note that the reconstructions span multiple languages. Add 'web agent' and 'frontier LLM' to the term library. > Web agents increasingly browse, shop, and manage accounts on behalf of their users. date — pass: originalPublished 2026-11-03 matches the v1 submission date. The v2 revision on 4 Nov and the 5 Nov feed entry are later, so they correctly do not set the date. > \[Submitted on 3 Nov 2026 (v1), last revised 4 Nov 2026 (this version, v2)\] support — pass: Each claim matches the abstract's wording, including the 11%–35% range, the payment-versus-credentials contrast and agents proceeding despite suspicion. The interpretation that model-specific rates are unavailable is correctly labeled and supported. The question follows from that gap without leading. The draft makes no unsupported claims about benefits or real-world reliability. > some agents acknowledge suspicious conditions yet proceed, at times treating the user's instructions as authorisation to accept additional terms imposed by the website. media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/adf217e21f047aac6f429293203d9524e05a1931b111f6ec7e4a0c23b8408124/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-paper-introduces-scamworld-to-test-web-agents-against-adf217e2.json)