# arXiv cs.AI: LUCA: A Synthetic World for Experimental Reasoning [Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-luca-a-synthetic-world-for-experimental-reasoning-1147e4bd) Authors Anna Gogleva, Thomas Charman, Martin Balla and Karl Gemayel introduce LUCA, a configurable synthetic world for testing how agents investigate unknown systems—not just whether they reach the right outcome. They argue that an agent can succeed for the wrong reasons when evaluations judge outcomes but overlook experiments, evidence and inferences. In LUCA, agents use limited experiments to learn regulatory systems, predict unseen interventions, diagnose hidden faults and attempt rescues. The authors report that these abilities can diverge: agents may rescue a system while identifying the wrong fault, and collecting evidence does not guarantee that an agent uses it. They also find that models adopt different experimental strategies, while more exploration does not reliably improve performance. The authors present LUCA as a controlled way to measure where experimental agents succeed and fail, separating skills that an outcome-only score could blur. The abstract reports results across seven frontier LLMs and three baselines, but does not establish how well these findings transfer beyond the synthetic world. How do agents’ experimental and diagnostic failures compare when evaluated on real systems with known ground truth? Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 2, 2026 (2026-11-02). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-10T08:53:02.218Z. First observed: 2026-11-03T09:04:18.757Z. Retrieved: 2026-11-04T02:46:09.827Z. - [LUCA: A Synthetic World for Experimental Reasoning](https://arxiv.org/abs/2611.03100) ## Limits The evidence is the paper’s abstract; it does not provide experimental details or results beyond the reported summary. The abstract does not establish whether findings from LUCA’s synthetic world transfer to real systems. ## Open question How do agents’ experimental and diagnostic failures compare when evaluated on real systems with known ground truth? ## Source-stated The authors introduce LUCA as a configurable synthetic world for evaluating agent experimentation and reasoning under known ground truth. > We introduce LUCA, a configurable synthetic world for evaluating this process under fully known ground truth. - [Supporting source](https://arxiv.org/abs/2611.03100) ## Source-stated The authors say evaluations often emphasize outcomes rather than the experiments, evidence and inferences behind them. > Evaluations, however, often focus on the outcome, not the experiments, evidence, and inferences that produced it. - [Supporting source](https://arxiv.org/abs/2611.03100) ## Source-stated The authors report that agents may rescue malfunctioning systems while diagnosing the wrong fault. > Agents frequently rescue malfunctioning systems while diagnosing the wrong fault; - [Supporting source](https://arxiv.org/abs/2611.03100) ## Source-stated The authors report that more exploration does not reliably lead to better performance. > greater exploration does not reliably yield better performance. - [Supporting source](https://arxiv.org/abs/2611.03100) ## Source-stated The abstract reports evaluation across seven frontier LLMs and three baselines. > Across seven frontier LLMs and three baselines, these capabilities dissociate sharply. - [Supporting source](https://arxiv.org/abs/2611.03100) ## Source and contributor context Gogleva, Anna, Charman, Thomas, Balla, Martin, Gemayel, Karl · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: a77f577f-7f94-410e-8309-9985ad3ca326. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T08:53:02.218Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: ad442d33ecd6b8708f1d35127ec65b240a8df924557ee3648601891412b82af4. Source text hash: 100507baf260ae17f77f75bf8655e9d95226846bff120431a71fcd0dc58568bf. ## Revision and source-check history ### Version 1 — 2026-11-10T08:53:02.218Z Authors Anna Gogleva, Thomas Charman, Martin Balla and Karl Gemayel introduce LUCA, a configurable synthetic world for testing how agents investigate unknown systems—not just whether they reach the right outcome. They argue that an agent can succeed for the wrong reasons when evaluations judge outcomes but overlook experiments, evidence and inferences. In LUCA, agents use limited experiments to learn regulatory systems, predict unseen interventions, diagnose hidden faults and attempt rescues. The authors report that these abilities can diverge: agents may rescue a system while identifying the wrong fault, and collecting evidence does not guarantee that an agent uses it. They also find that models adopt different experimental strategies, while more exploration does not reliably improve performance. The authors present LUCA as a controlled way to measure where experimental agents succeed and fail, separating skills that an outcome-only score could blur. The abstract reports results across seven frontier LLMs and three baselines, but does not establish how well these findings transfer beyond the synthetic world. How do agents’ experimental and diagnostic failures compare when evaluated on real systems with known ground truth? ## Original source arXiv cs.AI · Article Original source date: Nov 2, 2026 (2026-11-02). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-10T08:53:02.218Z. First observed: 2026-11-03T09:04:18.757Z. Retrieved: 2026-11-04T02:46:09.827Z. - [LUCA: A Synthetic World for Experimental Reasoning](https://arxiv.org/abs/2611.03100) ## Limits The evidence is the paper’s abstract; it does not provide experimental details or results beyond the reported summary. The abstract does not establish whether findings from LUCA’s synthetic world transfer to real systems. ## Open question How do agents’ experimental and diagnostic failures compare when evaluated on real systems with known ground truth? ## Source-stated The authors introduce LUCA as a configurable synthetic world for evaluating agent experimentation and reasoning under known ground truth. > We introduce LUCA, a configurable synthetic world for evaluating this process under fully known ground truth. - [Supporting source](https://arxiv.org/abs/2611.03100) ## Source-stated The authors say evaluations often emphasize outcomes rather than the experiments, evidence and inferences behind them. > Evaluations, however, often focus on the outcome, not the experiments, evidence, and inferences that produced it. - [Supporting source](https://arxiv.org/abs/2611.03100) ## Source-stated The authors report that agents may rescue malfunctioning systems while diagnosing the wrong fault. > Agents frequently rescue malfunctioning systems while diagnosing the wrong fault; - [Supporting source](https://arxiv.org/abs/2611.03100) ## Source-stated The authors report that more exploration does not reliably lead to better performance. > greater exploration does not reliably yield better performance. - [Supporting source](https://arxiv.org/abs/2611.03100) ## Source-stated The abstract reports evaluation across seven frontier LLMs and three baselines. > Across seven frontier LLMs and three baselines, these capabilities dissociate sharply. - [Supporting source](https://arxiv.org/abs/2611.03100) ## Source and contributor context Gogleva, Anna, Charman, Thomas, Balla, Martin, Gemayel, Karl · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: a77f577f-7f94-410e-8309-9985ad3ca326. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T08:53:02.218Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: ad442d33ecd6b8708f1d35127ec65b240a8df924557ee3648601891412b82af4. Source text hash: 100507baf260ae17f77f75bf8655e9d95226846bff120431a71fcd0dc58568bf. ### uncertain — 2026-11-10T08:53:02.218Z — anthropic/claude-opus-5.5 Publication: a77f577f-7f94-410e-8309-9985ad3ca326 attribution — pass: The draft correctly credits the four listed authors and keeps the findings framed as what the authors introduce, argue and report. The reading does not present them as independently verified facts. > Authors:Anna Gogleva, Thomas Charman, Martin Balla, Karl Gemayel summary — uncertain: Two passages need repair. First, 'regulatory systems' is central but unexplained; a general reader could take it to mean legal or policy regulation. The abstract describes unknown systems that agents probe through experiments. Add a tooltip or plain-language gloss without inventing a domain. Second, the draft softens the authors' frequency claims: 'frequently' becomes 'may', and 'often perform the experiment... but still fail' becomes 'does not guarantee'. It also drops the striking 7x versus 1.4x contrast. Restoring 'frequently', 'often' and that ratio would preserve the authors' force. Terms such as frontier LLMs, baselines and ground truth would benefit from tooltips. > Agents frequently rescue malfunctioning systems while diagnosing the wrong fault; diagnosis varies 7x across models while prediction varies only 1.4x. date — pass: The originalPublished date, 2026-11-02, matches the v1 submission date. The later feed date of November 3 is correctly not used. > \[Submitted on 2 Nov 2026\] support — pass: Every claim matches a verbatim excerpt from the abstract. The stated limitation that transfer beyond the synthetic world is not established is a fair gap rather than an invented claim. The question follows from that gap and does not assert an answer. > In one failure mode, agents often perform the experiment needed to distinguish competing causes, but still fail to use the resulting evidence. media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/1147e4bd57454d9e57f3e5fedff6924b1b3854f7274d7e45abafce998c22e36b/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-luca-a-synthetic-world-for-experimental-reasoning-1147e4bd.json)