Wu, Xiaodong, Zhao, Zhimin, Li, Qi, Cheong, Chaklam, Li, Xiangman, Ni, Jianbing · Nov 5, 2026
What the Middleman Delivers: How LLM API relays compare
Markdown · JSON · Original source
Authors Xiaodong Wu, Zhimin Zhao, Qi Li, Chaklam Cheong, Xiangman Li and Jianbing Ni examine third-party relays, services that provide access to . They ask whether relays deliver expected model behavior, respect tool permissions and bill accurately—not just whether their advertised models and prices look attractive. The authors compare answers from 20 relays offering GPT-6 Astra and Opus 5 with batch-matched official responses. In the first-round sample, their analysis flags behavioral differences at 7/10 low-price relays and 2/10 same- or higher-price relays. Retesting changes some flags, so a single comparison may not give a stable picture. They report no confirmed injection in 3,332 screened responses to a fixed, non-adversarial code and setup workload. But four relays return tool calls despite instructions disabling them. For identical requests, five relays report 4,200 to 60,000 more than the official API. Where balances are available, charges follow reported usage at listed prices, including extra reported input tokens at one service. The authors argue that advertised models and prices are insufficient to judge relay reliability. They identify unexplained billed token quantities and tool-permission violations as risks, and say service evaluations need repeated measurements.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 5, 2026 (2026-11-05).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-08T03:18:44.244Z. First observed: 2026-11-06T05:10:45.324Z. Retrieved: 2026-11-08T03:17:36.884Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The reported tests cover 20 relays and a fixed, non-adversarial code and setup workload for the injection screening; the abstract does not establish how results generalize beyond the tested services and conditions. The abstract reports retest changes in behavioral flags, so the initial comparison does not alone establish that those differences persist across measurements.
Open question
How consistently do the reported behavioral differences appear across repeated measurements?
Source-stated
The authors compared 20 relays offering GPT-6 Astra and Claude Opus 5 against batch-matched official responses.
We test 20 relays offering GPT-6 Astra and Claude Opus 5. First, we compare answers to knowledge questions with batch-matched official responses.
Source-stated
In the first-round sample, the analysis flagged behavioral differences at 7/10 low-price relays and 2/10 same- or higher-price relays; retesting changed some flags.
In our first-round sample, the analysis flags 7/10 low-price and 2/10 same- or higher-price relays for behavioral differences. On retesting 35 pairs, seven gain or lose a flag under the original rules
Source-stated
The authors report no confirmed injection in the screened responses, but four relays returned tool calls despite instructions disabling them.
No injection is confirmed in 3,332 screened responses to a fixed, non-adversarial code and setup workload, yet four relays return tool calls despite instructions disabling them in the first round.
Source-stated
For identical requests, five relays reported substantially more input tokens than the official API; where balances were available, charges followed reported usage at listed prices.
For identical requests, five relays report 4,200 to 60,000 more input tokens than the official API. Where balances are available (six services), charges follow reported usage at listed prices
Source-stated
The authors argue that advertised models and prices alone are insufficient to judge relay reliability.
Our study shows that advertised models and prices are insufficient to judge relay reliability.
Source and contributor context
Wu, Xiaodong, Zhao, Zhimin, Li, Qi, Cheong, Chaklam, Li, Xiangman, Ni, Jianbing · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
API
An application programming interface: a defined way for software to request data or actions from another system.
Model tokens
Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Claude
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Editorial provenance
Version 1. Publication ID: a3ad766d-5d81-4fef-8040-3e7af776f2ef.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-08T03:18:44.244Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: b6d105ef882e58d32525648583bb35522574dff2c57b5385287094095275174e. Source text hash: f25e3d1d8f37bd664414f6b06ffb8449a21f0a292010ea468fa132a693891db7.
Revision and source-check history
Version 1 · 2026-11-08T03:18:44.244Z
What the Middleman Delivers: How LLM API relays compare
Authors Xiaodong Wu, Zhimin Zhao, Qi Li, Chaklam Cheong, Xiangman Li and Jianbing Ni examine third-party relays, services that provide access to . They ask whether relays deliver expected model behavior, respect tool permissions and bill accurately—not just whether their advertised models and prices look attractive. The authors compare answers from 20 relays offering GPT-6 Astra and Opus 5 with batch-matched official responses. In the first-round sample, their analysis flags behavioral differences at 7/10 low-price relays and 2/10 same- or higher-price relays. Retesting changes some flags, so a single comparison may not give a stable picture. They report no confirmed injection in 3,332 screened responses to a fixed, non-adversarial code and setup workload. But four relays return tool calls despite instructions disabling them. For identical requests, five relays report 4,200 to 60,000 more than the official API. Where balances are available, charges follow reported usage at listed prices, including extra reported input tokens at one service. The authors argue that advertised models and prices are insufficient to judge relay reliability. They identify unexplained billed token quantities and tool-permission violations as risks, and say service evaluations need repeated measurements.
Original source
cs.AI · Article
Source details
Original source date: Nov 5, 2026 (2026-11-05).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-08T03:18:44.244Z. First observed: 2026-11-06T05:10:45.324Z. Retrieved: 2026-11-08T03:17:36.884Z.
Limits
The reported tests cover 20 relays and a fixed, non-adversarial code and setup workload for the injection screening; the abstract does not establish how results generalize beyond the tested services and conditions. The abstract reports retest changes in behavioral flags, so the initial comparison does not alone establish that those differences persist across measurements.
Open question
How consistently do the reported behavioral differences appear across repeated measurements?
Source-stated
The authors compared 20 relays offering GPT-6 Astra and Claude Opus 5 against batch-matched official responses.
We test 20 relays offering GPT-6 Astra and Claude Opus 5. First, we compare answers to knowledge questions with batch-matched official responses.
Source-stated
In the first-round sample, the analysis flagged behavioral differences at 7/10 low-price relays and 2/10 same- or higher-price relays; retesting changed some flags.
In our first-round sample, the analysis flags 7/10 low-price and 2/10 same- or higher-price relays for behavioral differences. On retesting 35 pairs, seven gain or lose a flag under the original rules
Source-stated
The authors report no confirmed injection in the screened responses, but four relays returned tool calls despite instructions disabling them.
No injection is confirmed in 3,332 screened responses to a fixed, non-adversarial code and setup workload, yet four relays return tool calls despite instructions disabling them in the first round.
Source-stated
For identical requests, five relays reported substantially more input tokens than the official API; where balances were available, charges followed reported usage at listed prices.
For identical requests, five relays report 4,200 to 60,000 more input tokens than the official API. Where balances are available (six services), charges follow reported usage at listed prices
Source-stated
The authors argue that advertised models and prices alone are insufficient to judge relay reliability.
Our study shows that advertised models and prices are insufficient to judge relay reliability.
Source and contributor context
Wu, Xiaodong, Zhao, Zhimin, Li, Qi, Cheong, Chaklam, Li, Xiangman, Ni, Jianbing · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
API
An application programming interface: a defined way for software to request data or actions from another system.
Model tokens
Small chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Claude
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Editorial provenance
Version 1. Publication ID: a3ad766d-5d81-4fef-8040-3e7af776f2ef.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-08T03:18:44.244Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: b6d105ef882e58d32525648583bb35522574dff2c57b5385287094095275174e. Source text hash: f25e3d1d8f37bd664414f6b06ffb8449a21f0a292010ea468fa132a693891db7.
uncertain · 2026-11-08T03:18:44.244Z · anthropic/claude-opus-5.5
Publication: a3ad766d-5d81-4fef-8040-3e7af776f2ef
attribution — pass: The draft correctly attributes the study to Xiaodong Wu, Zhimin Zhao, Qi Li, Chaklam Cheong, Xiangman Li and Jianbing Ni. It keeps the findings and conclusions framed as the authors' reported results and arguments.
Authors:Xiaodong Wu, Zhimin Zhao, Qi Li, Chaklam Cheong, Xiangman Li, Jianbing Ni
summary — uncertain: The structure is faithful and the question follows from the evidence. A non-specialist, however, meets central terms without explanation. 'Injection' is never explained, so 'no confirmed injection' is opaque; 'tool calls', 'input tokens' and 'batch-matched' are also left bare. The definition of relays also drops 'often at lower prices', which drives the low-price versus same/higher-price comparison. Smallest repair: restore the lower-price context and add term-library tooltips for injection, tool calls, input tokens and batch-matched, keeping the terms themselves.
Third-party application programming interface (API) relays offer access to large language models (LLMs), often at lower prices.
date — pass: originalPublished 2026-11-05 matches the v1 submission date on arXiv. The feed date of Nov 6 is later and correctly not used.
[Submitted on 5 Nov 2026]
support — uncertain: Most claims match the abstract. However, the summary and claim 3 drop the scope condition 'in the first round' from the tool-call finding, presenting it as unqualified. This matters because the paper stresses that retesting changes results. Smallest repair: add 'in the first round' to that sentence. Retest figures (35 pairs; seven flag changes under the original rules, six under matched corrections) are also compressed to 'changes some flags'. That compression is acceptable but less precise.
yet four relays return tool calls despite instructions disabling them in the first round.
media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source and the most relevant access point.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
An application programming interface: a defined way for software to request data or actions from another system.
Definition referenceA language model with many learned numerical settings, used to process and generate language.
Definition referenceClaude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Explore ClaudeSmall chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivAn application programming interface: a defined way for software to request data or actions from another system.
Definition referenceA language model with many learned numerical settings, used to process and generate language.
Definition referenceClaude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Explore ClaudeSmall chunks of text, often parts of words, that a language model reads and writes; usage and prices are often counted in tokens.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv