# What the Middleman Delivers: How LLM API relays compare [Canonical reading](https://escapemeridian.com/readings/what-the-middleman-delivers-how-llm-api-relays-compare-e04cb86c) Authors Xiaodong Wu, Zhimin Zhao, Qi Li, Chaklam Cheong, Xiangman Li and Jianbing Ni examine third-party API relays, services that provide access to large language models. They ask whether relays deliver expected model behavior, respect tool permissions and bill accurately—not just whether their advertised models and prices look attractive. The authors compare answers from 20 relays offering GPT-6 Astra and Claude Opus 5 with batch-matched official responses. In the first-round sample, their analysis flags behavioral differences at 7/10 low-price relays and 2/10 same- or higher-price relays. Retesting changes some flags, so a single comparison may not give a stable picture. They report no confirmed injection in 3,332 screened responses to a fixed, non-adversarial code and setup workload. But four relays return tool calls despite instructions disabling them. For identical requests, five relays report 4,200 to 60,000 more input tokens than the official API. Where balances are available, charges follow reported usage at listed prices, including extra reported input tokens at one service. The authors argue that advertised models and prices are insufficient to judge relay reliability. They identify unexplained billed token quantities and tool-permission violations as risks, and say service evaluations need repeated measurements. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 5, 2026 (2026-11-05). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-08T03:18:44.244Z. First observed: 2026-11-06T05:10:45.324Z. Retrieved: 2026-11-08T03:17:36.884Z. - [What the Middleman Delivers: Answer Behavior, Tool Permissions, and Billed Usage in LLM API Relays](https://arxiv.org/abs/2611.07650) ## Limits The reported tests cover 20 relays and a fixed, non-adversarial code and setup workload for the injection screening; the abstract does not establish how results generalize beyond the tested services and conditions. The abstract reports retest changes in behavioral flags, so the initial comparison does not alone establish that those differences persist across measurements. ## Open question How consistently do the reported behavioral differences appear across repeated measurements? ## Source-stated The authors compared 20 relays offering GPT-6 Astra and Claude Opus 5 against batch-matched official responses. > We test 20 relays offering GPT-6 Astra and Claude Opus 5. First, we compare answers to knowledge questions with batch-matched official responses. - [Supporting source](https://arxiv.org/abs/2611.07650) ## Source-stated In the first-round sample, the analysis flagged behavioral differences at 7/10 low-price relays and 2/10 same- or higher-price relays; retesting changed some flags. > In our first-round sample, the analysis flags 7/10 low-price and 2/10 same- or higher-price relays for behavioral differences. On retesting 35 pairs, seven gain or lose a flag under the original rules - [Supporting source](https://arxiv.org/abs/2611.07650) ## Source-stated The authors report no confirmed injection in the screened responses, but four relays returned tool calls despite instructions disabling them. > No injection is confirmed in 3,332 screened responses to a fixed, non-adversarial code and setup workload, yet four relays return tool calls despite instructions disabling them in the first round. - [Supporting source](https://arxiv.org/abs/2611.07650) ## Source-stated For identical requests, five relays reported substantially more input tokens than the official API; where balances were available, charges followed reported usage at listed prices. > For identical requests, five relays report 4,200 to 60,000 more input tokens than the official API. Where balances are available (six services), charges follow reported usage at listed prices - [Supporting source](https://arxiv.org/abs/2611.07650) ## Source-stated The authors argue that advertised models and prices alone are insufficient to judge relay reliability. > Our study shows that advertised models and prices are insufficient to judge relay reliability. - [Supporting source](https://arxiv.org/abs/2611.07650) ## Source and contributor context Wu, Xiaodong, Zhao, Zhimin, Li, Qi, Cheong, Chaklam, Li, Xiangman, Ni, Jianbing · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## API An application programming interface: a defined way for software to request data or actions from another system. - [Definition reference](https://developer.mozilla.org/en-US/docs/Glossary/API) ## Model tokens Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens. - [Definition reference](https://developers.google.com/machine-learning/glossary#token) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Claude Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding. - [Explore this context](/entities/claude) - [Reference](https://claude.com/product/overview) - [Reference](https://www.anthropic.com/news) - [Reference](https://www.anthropic.com/research/team/economics) ## Editorial provenance Version 1. Publication ID: a3ad766d-5d81-4fef-8040-3e7af776f2ef. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-08T03:18:44.244Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: b6d105ef882e58d32525648583bb35522574dff2c57b5385287094095275174e. Source text hash: f25e3d1d8f37bd664414f6b06ffb8449a21f0a292010ea468fa132a693891db7. ## Revision and source-check history ### Version 1 — 2026-11-08T03:18:44.244Z Authors Xiaodong Wu, Zhimin Zhao, Qi Li, Chaklam Cheong, Xiangman Li and Jianbing Ni examine third-party API relays, services that provide access to large language models. They ask whether relays deliver expected model behavior, respect tool permissions and bill accurately—not just whether their advertised models and prices look attractive. The authors compare answers from 20 relays offering GPT-6 Astra and Claude Opus 5 with batch-matched official responses. In the first-round sample, their analysis flags behavioral differences at 7/10 low-price relays and 2/10 same- or higher-price relays. Retesting changes some flags, so a single comparison may not give a stable picture. They report no confirmed injection in 3,332 screened responses to a fixed, non-adversarial code and setup workload. But four relays return tool calls despite instructions disabling them. For identical requests, five relays report 4,200 to 60,000 more input tokens than the official API. Where balances are available, charges follow reported usage at listed prices, including extra reported input tokens at one service. The authors argue that advertised models and prices are insufficient to judge relay reliability. They identify unexplained billed token quantities and tool-permission violations as risks, and say service evaluations need repeated measurements. ## Original source arXiv cs.AI · Article Original source date: Nov 5, 2026 (2026-11-05). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-08T03:18:44.244Z. First observed: 2026-11-06T05:10:45.324Z. Retrieved: 2026-11-08T03:17:36.884Z. - [What the Middleman Delivers: Answer Behavior, Tool Permissions, and Billed Usage in LLM API Relays](https://arxiv.org/abs/2611.07650) ## Limits The reported tests cover 20 relays and a fixed, non-adversarial code and setup workload for the injection screening; the abstract does not establish how results generalize beyond the tested services and conditions. The abstract reports retest changes in behavioral flags, so the initial comparison does not alone establish that those differences persist across measurements. ## Open question How consistently do the reported behavioral differences appear across repeated measurements? ## Source-stated The authors compared 20 relays offering GPT-6 Astra and Claude Opus 5 against batch-matched official responses. > We test 20 relays offering GPT-6 Astra and Claude Opus 5. First, we compare answers to knowledge questions with batch-matched official responses. - [Supporting source](https://arxiv.org/abs/2611.07650) ## Source-stated In the first-round sample, the analysis flagged behavioral differences at 7/10 low-price relays and 2/10 same- or higher-price relays; retesting changed some flags. > In our first-round sample, the analysis flags 7/10 low-price and 2/10 same- or higher-price relays for behavioral differences. On retesting 35 pairs, seven gain or lose a flag under the original rules - [Supporting source](https://arxiv.org/abs/2611.07650) ## Source-stated The authors report no confirmed injection in the screened responses, but four relays returned tool calls despite instructions disabling them. > No injection is confirmed in 3,332 screened responses to a fixed, non-adversarial code and setup workload, yet four relays return tool calls despite instructions disabling them in the first round. - [Supporting source](https://arxiv.org/abs/2611.07650) ## Source-stated For identical requests, five relays reported substantially more input tokens than the official API; where balances were available, charges followed reported usage at listed prices. > For identical requests, five relays report 4,200 to 60,000 more input tokens than the official API. Where balances are available (six services), charges follow reported usage at listed prices - [Supporting source](https://arxiv.org/abs/2611.07650) ## Source-stated The authors argue that advertised models and prices alone are insufficient to judge relay reliability. > Our study shows that advertised models and prices are insufficient to judge relay reliability. - [Supporting source](https://arxiv.org/abs/2611.07650) ## Source and contributor context Wu, Xiaodong, Zhao, Zhimin, Li, Qi, Cheong, Chaklam, Li, Xiangman, Ni, Jianbing · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## API An application programming interface: a defined way for software to request data or actions from another system. - [Definition reference](https://developer.mozilla.org/en-US/docs/Glossary/API) ## Model tokens Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens. - [Definition reference](https://developers.google.com/machine-learning/glossary#token) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Claude Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding. - [Explore this context](/entities/claude) - [Reference](https://claude.com/product/overview) - [Reference](https://www.anthropic.com/news) - [Reference](https://www.anthropic.com/research/team/economics) ## Editorial provenance Version 1. Publication ID: a3ad766d-5d81-4fef-8040-3e7af776f2ef. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-08T03:18:44.244Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: b6d105ef882e58d32525648583bb35522574dff2c57b5385287094095275174e. Source text hash: f25e3d1d8f37bd664414f6b06ffb8449a21f0a292010ea468fa132a693891db7. ### uncertain — 2026-11-08T03:18:44.244Z — anthropic/claude-opus-5.5 Publication: a3ad766d-5d81-4fef-8040-3e7af776f2ef attribution — pass: The draft correctly attributes the study to Xiaodong Wu, Zhimin Zhao, Qi Li, Chaklam Cheong, Xiangman Li and Jianbing Ni. It keeps the findings and conclusions framed as the authors' reported results and arguments. > Authors:Xiaodong Wu, Zhimin Zhao, Qi Li, Chaklam Cheong, Xiangman Li, Jianbing Ni summary — uncertain: The structure is faithful and the question follows from the evidence. A non-specialist, however, meets central terms without explanation. 'Injection' is never explained, so 'no confirmed injection' is opaque; 'tool calls', 'input tokens' and 'batch-matched' are also left bare. The definition of relays also drops 'often at lower prices', which drives the low-price versus same/higher-price comparison. Smallest repair: restore the lower-price context and add term-library tooltips for injection, tool calls, input tokens and batch-matched, keeping the terms themselves. > Third-party application programming interface (API) relays offer access to large language models (LLMs), often at lower prices. date — pass: originalPublished 2026-11-05 matches the v1 submission date on arXiv. The feed date of Nov 6 is later and correctly not used. > \[Submitted on 5 Nov 2026\] support — uncertain: Most claims match the abstract. However, the summary and claim 3 drop the scope condition 'in the first round' from the tool-call finding, presenting it as unqualified. This matters because the paper stresses that retesting changes results. Smallest repair: add 'in the first round' to that sentence. Retest figures (35 pairs; seven flag changes under the original rules, six under matched corrections) are also compressed to 'changes some flags'. That compression is acceptable but less precise. > yet four relays return tool calls despite instructions disabling them in the first round. media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source and the most relevant access point. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/e04cb86cebbe2585a8ce32927ebd149fb06d43126eb3ab6770097b8bdd786bba/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/what-the-middleman-delivers-how-llm-api-relays-compare-e04cb86c.json)