Amar, Sara · Nov 17, 2026
Sara Amar on MANGO Arena: same seeds, different chains
Markdown · JSON · Original source
Sara Amar introduces MANGO Arena, an open-source testbed where four agents run companies in a serial supply chain. It tracks the bullwhip effect—demand variability growing as orders move upstream—as well as rule-following and proposals to collude. In a pilot using three small , demand variability grew substantially upstream, often more than under a textbook ordering policy. Rule violations were rare, and no agent proposed a money side deal. But Amar’s central warning is about evaluation: failed calls were silently counted as “no order,” and even reruns with the same seeds and no failed calls produced large changes in upstream bullwhip. With only three demand seeds, conclusions about a disruption’s effect on a model did not survive rerunning the games. The paper reports these failures and proposes reporting practices for similar evaluations; its code, prompts and logged decisions are public. The pilot therefore illustrates how both system failures and run-to-run variation can shape apparent agent performance, while leaving the stability of findings beyond this small test unresolved.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 17, 2026 (2026-11-17).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-19T07:31:54.748Z. First observed: 2026-11-19T05:04:18.159Z. Retrieved: 2026-11-19T07:30:59.037Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The findings come from a pilot with three small , two scripted reference policies, two conditions, and three demand seeds; they do not establish how results would generalize beyond those tests. The source reports that failures affected metrics and that reruns varied, so the pilot's performance measurements require caution.
Open question
How stable are the reported findings when the games are rerun with more demand seeds and without failed calls?
Source-stated
Sara Amar introduces MANGO Arena, an open-source testbed where four LLM agents each operate a company in a serial supply chain.
We introduce MANGO Arena, an open-source testbed in which four LLM agents each run one company in a serial supply chain
Source-stated
MANGO Arena measures the bullwhip effect, rule adherence, and proposals to collude.
This lets one environment measure three things at once: an economic outcome (the bullwhip effect), rule adherence under pressure, and proposals to collude.
Source-stated
In a pilot, all LLM agents amplified demand variability upstream, often more than a textbook order-up-to policy.
all LLM agents amplified demand variability as it moved upstream, by 7 to 25 times at the Retailer and 13 to 179 times at the Supplier, often more than a textbook order-up-to policy.
Source-stated
The pilot recorded rare rule violations and no proposed money side deals.
Real rule violations were rare (at most 7 of 288 decisions per model) and no agent proposed a money side deal.
Source-stated
The authors report that API failures silently counted as “no order” decisions, while upstream bullwhip varied substantially between reruns.
API failures (rate limits and runaway hidden reasoning) entered the metrics silently as "no order" decisions. And when we reran games with the same seeds and no failed calls, upstream bullwhip changed by up to 17 times between runs.
Source-stated
The authors say that conclusions about disruption effects did not survive reruns with only three demand seeds.
With three seeds, conclusions about how a disruption affects a given model did not survive a rerun.
Source and contributor context
Amar, Sara · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Reasoning model
A language model trained to work through a problem in written steps before it answers.
API
An application programming interface: a defined way for software to request data or actions from another system.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: d634675f-1fbf-495f-85b0-6b3e2e387e41.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-19T07:31:54.748Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: b0360fabe99f6a6fc79cc864523f390be055f91c1040a7d0f22d74f238494da5. Source text hash: 052df621fa86eee29d87232eaf08e8e210208187a9281be3e6071840ae2b4950.
Revision and source-check history
Version 1 · 2026-11-19T07:31:54.748Z
Sara Amar on MANGO Arena: same seeds, different chains
Sara Amar introduces MANGO Arena, an open-source testbed where four agents run companies in a serial supply chain. It tracks the bullwhip effect—demand variability growing as orders move upstream—as well as rule-following and proposals to collude. In a pilot using three small , demand variability grew substantially upstream, often more than under a textbook ordering policy. Rule violations were rare, and no agent proposed a money side deal. But Amar’s central warning is about evaluation: failed calls were silently counted as “no order,” and even reruns with the same seeds and no failed calls produced large changes in upstream bullwhip. With only three demand seeds, conclusions about a disruption’s effect on a model did not survive rerunning the games. The paper reports these failures and proposes reporting practices for similar evaluations; its code, prompts and logged decisions are public. The pilot therefore illustrates how both system failures and run-to-run variation can shape apparent agent performance, while leaving the stability of findings beyond this small test unresolved.
Original source
cs.AI · Article
Source details
Original source date: Nov 17, 2026 (2026-11-17).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-19T07:31:54.748Z. First observed: 2026-11-19T05:04:18.159Z. Retrieved: 2026-11-19T07:30:59.037Z.
Limits
The findings come from a pilot with three small , two scripted reference policies, two conditions, and three demand seeds; they do not establish how results would generalize beyond those tests. The source reports that failures affected metrics and that reruns varied, so the pilot's performance measurements require caution.
Open question
How stable are the reported findings when the games are rerun with more demand seeds and without failed calls?
Source-stated
Sara Amar introduces MANGO Arena, an open-source testbed where four LLM agents each operate a company in a serial supply chain.
We introduce MANGO Arena, an open-source testbed in which four LLM agents each run one company in a serial supply chain
Source-stated
MANGO Arena measures the bullwhip effect, rule adherence, and proposals to collude.
This lets one environment measure three things at once: an economic outcome (the bullwhip effect), rule adherence under pressure, and proposals to collude.
Source-stated
In a pilot, all LLM agents amplified demand variability upstream, often more than a textbook order-up-to policy.
all LLM agents amplified demand variability as it moved upstream, by 7 to 25 times at the Retailer and 13 to 179 times at the Supplier, often more than a textbook order-up-to policy.
Source-stated
The pilot recorded rare rule violations and no proposed money side deals.
Real rule violations were rare (at most 7 of 288 decisions per model) and no agent proposed a money side deal.
Source-stated
The authors report that API failures silently counted as “no order” decisions, while upstream bullwhip varied substantially between reruns.
API failures (rate limits and runaway hidden reasoning) entered the metrics silently as "no order" decisions. And when we reran games with the same seeds and no failed calls, upstream bullwhip changed by up to 17 times between runs.
Source-stated
The authors say that conclusions about disruption effects did not survive reruns with only three demand seeds.
With three seeds, conclusions about how a disruption affects a given model did not survive a rerun.
Source and contributor context
Amar, Sara · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Reasoning model
A language model trained to work through a problem in written steps before it answers.
API
An application programming interface: a defined way for software to request data or actions from another system.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: d634675f-1fbf-495f-85b0-6b3e2e387e41.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-19T07:31:54.748Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: b0360fabe99f6a6fc79cc864523f390be055f91c1040a7d0f22d74f238494da5. Source text hash: 052df621fa86eee29d87232eaf08e8e210208187a9281be3e6071840ae2b4950.
uncertain · 2026-11-19T07:31:54.748Z · anthropic/claude-opus-5.5
Publication: d634675f-1fbf-495f-85b0-6b3e2e387e41
attribution — pass: The paper is correctly attributed to Sara Amar, its sole listed author, on arXiv. The claims say 'the authors,' while the summary correctly uses Amar. This is a minor wording inconsistency and does not misattribute any claim.
Authors:Sara Amar
summary — uncertain: The summary is faithful and keeps the paper's central point about evaluation method. Two gaps limit understanding for non-specialists. First, 'seeds' and 'demand seeds' are never explained, yet the main lesson depends on them: the same fixed demand scenarios, rerun, gave very different results. Second, the 'disruption' is never identified as a supplier fire, so 'a disruption's effect' lacks context. Smallest repair: add a short gloss on seeds and name the supplier-fire condition. 'Order-up-to policy' could also use a term tooltip.
two conditions (with and without a supplier fire) and three demand seeds
date — pass: The original publication date of 2026-11-17 matches the v1 submission date. The feed date of Nov 19 is correctly not used.
[Submitted on 17 Nov 2026]
support — pass: Every claim is directly supported by the abstract. The closing interpretation, that system failures and run-to-run variation can shape apparent performance, follows from what the source reports. The limitations and the question are framed appropriately and do not turn the pilot into a general finding.
With three seeds, conclusions about how a disruption affects a given model did not survive a rerun.
media — pass: The selected media is the verified arXiv abstract page itself, which is the direct original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A language model with many learned numerical settings, used to process and generate language.
Definition referenceA language model trained to work through a problem in written steps before it answers.
Definition referenceAn application programming interface: a defined way for software to request data or actions from another system.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA language model trained to work through a problem in written steps before it answers.
Definition referenceAn application programming interface: a defined way for software to request data or actions from another system.
Definition referenceAn application programming interface: a defined way for software to request data or actions from another system.
Definition referenceA language model with many learned numerical settings, used to process and generate language.
Definition referenceA language model trained to work through a problem in written steps before it answers.
Definition referenceAn application programming interface: a defined way for software to request data or actions from another system.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA language model trained to work through a problem in written steps before it answers.
Definition referenceAn application programming interface: a defined way for software to request data or actions from another system.
Definition referenceAn application programming interface: a defined way for software to request data or actions from another system.
Definition reference