{"id":"1bb3b466765fb7db5fc6edc5c9c98c05eb26528ea7dc12aa0acfc344fef8dd66","title":"Rakshit S Trivedi and co-authors argue in arXiv for cooperative AI design","argument":"The authors’ work was published on June 4, 2026. Rakshit S Trivedi, Natasha Jaques, Logan Cross, Alexander Sasha Vezhnevets and Joel Z Leibo argue that a superintelligence developed as a solipsistic task solver is unlikely to cooperate. They say the usual approach treats the world as a fixed source of feedback, but deploying AI systems changes the conditions those systems encounter. They call this mismatch the “self-undermining property of unilateral optimization.” In their account, cooperation is not simply another task for AI to solve: it is how multiple actors navigate their interdependence. The authors call for research that makes this interdependence central to AI design, including evaluations with adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of AI systems. Their argument shifts attention from capability alone to coexistence, and makes human agency part of the design question. The abstract does not specify how these proposals would be implemented or demonstrate that they would make AI systems cooperative.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on November 2, 2026; originally published June 4, 2026. The available evidence is the publication page’s abstract; it does not provide details of the proposed methods or evidence that they would produce cooperative systems.","speaker":{"name":"Google DeepMind","roleAtSource":"Publisher"},"source":{"title":"Solipsistic superintelligence is unlikely to be cooperative","publisher":"Google DeepMind","format":"Article","url":"https://deepmind.google/research/publications/231466/","publishedAt":"2026-06-04T00:00:00+00:00","dateBasis":"Date stated on the publisher’s page."},"evidence":[{"text":"The authors argue that superintelligence developed through a solipsistic approach to AI design is unlikely to be cooperative.","excerpt":"superintelligence, an extremely capable task solver, born out of such a solipsistic approach to AI design, is unlikely to be cooperative.","url":"https://deepmind.google/research/publications/231466/","relationship":"Source-stated"},{"text":"The authors say deploying AI systems changes the conditions they encounter, creating a gap between historical distributions and deployment context.","excerpt":"Deploying AI systems induces endogenous non-stationarity, resulting in a train-test-deploy gap where historical distributions diverge from the deployment context.","url":"https://deepmind.google/research/publications/231466/","relationship":"Source-stated"},{"text":"The authors define cooperation as an equilibrium-selection process through which multiple actors navigate interdependence.","excerpt":"the equilibrium-selection process through which multiple actors navigate their interdependence.","url":"https://deepmind.google/research/publications/231466/","relationship":"Source-stated"},{"text":"The authors call for a research paradigm that treats interdependence as a core design principle, with adaptive evaluations, institutions as design primitives, and human agency as a structural feature.","excerpt":"This entails building dynamic evaluation testbeds involving adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of the systems we build.","url":"https://deepmind.google/research/publications/231466/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://deepmind.google/research/publications/231466/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How would dynamic evaluations involving adaptive counterparties show whether an AI system can participate in cooperation?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Rakshit S Trivedi and co-authors argue in arXiv for cooperative AI design","summary":"The authors’ work was published on June 4, 2026. Rakshit S Trivedi, Natasha Jaques, Logan Cross, Alexander Sasha Vezhnevets and Joel Z Leibo argue that a superintelligence developed as a solipsistic task solver is unlikely to cooperate. They say the usual approach treats the world as a fixed source of feedback, but deploying AI systems changes the conditions those systems encounter. They call this mismatch the “self-undermining property of unilateral optimization.” In their account, cooperation is not simply another task for AI to solve: it is how multiple actors navigate their interdependence. The authors call for research that makes this interdependence central to AI design, including evaluations with adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of AI systems. Their argument shifts attention from capability alone to coexistence, and makes human agency part of the design question. The abstract does not specify how these proposals would be implemented or demonstrate that they would make AI systems cooperative.","question":"How would dynamic evaluations involving adaptive counterparties show whether an AI system can participate in cooperation?","claims":[{"text":"The authors argue that superintelligence developed through a solipsistic approach to AI design is unlikely to be cooperative.","excerpt":"superintelligence, an extremely capable task solver, born out of such a solipsistic approach to AI design, is unlikely to be cooperative.","support":"source-stated"},{"text":"The authors say deploying AI systems changes the conditions they encounter, creating a gap between historical distributions and deployment context.","excerpt":"Deploying AI systems induces endogenous non-stationarity, resulting in a train-test-deploy gap where historical distributions diverge from the deployment context.","support":"source-stated"},{"text":"The authors define cooperation as an equilibrium-selection process through which multiple actors navigate interdependence.","excerpt":"the equilibrium-selection process through which multiple actors navigate their interdependence.","support":"source-stated"},{"text":"The authors call for a research paradigm that treats interdependence as a core design principle, with adaptive evaluations, institutions as design primitives, and human agency as a structural feature.","excerpt":"This entails building dynamic evaluation testbeds involving adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of the systems we build.","support":"source-stated"}],"limitations":["Text as retrieved on November 2, 2026; originally published June 4, 2026.","The available evidence is the publication page’s abstract; it does not provide details of the proposed methods or evidence that they would produce cooperative systems."],"mediaId":null,"correctionReason":null,"id":"1bb3b466765fb7db5fc6edc5c9c98c05eb26528ea7dc12aa0acfc344fef8dd66","publicationId":"4b4b3318-3306-65a0-12f8-eca53f06d12a","version":1,"sourceUrl":"https://deepmind.google/research/publications/231466/","sourceTitle":"Solipsistic superintelligence is unlikely to be cooperative","publisher":"Google DeepMind","publishedAt":"2026-06-04T00:00:00+00:00","originalPublished":{"value":"2026-06-04T00:00:00+00:00","precision":"instant","basis":"backfill:article_published"},"firstObservedAt":"2026-11-02T23:25:57.408Z","createdAt":"2026-11-06T22:08:24.340Z","historical":true,"entityIds":["google-deepmind"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://deepmind.google/research/publications/231466/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"0f3b32058535781e641b658d37e1fffd910c50c7476afefb01280b8191795cdd","rubricVersion":"unrecorded","provider":"anthropic","model":"claude-opus-5-5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-11-02T23:25:54.324Z"},"provenance":{"textHash":"c569e2f4d6d5c45f1790602b3465b1f9fa35af104ebb5df396d0bf3ae26d93f9","observationId":"39b65c32f6dcc23587e7dda8301488021a8f60fc2afe14f12022615999f9cdb9","retrievedAt":"2026-11-02T23:27:21.125Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"d5c85510-db0c-a3cb-64fb-e22fbd019f00"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"cfb25d1b-e9b2-6a66-9b38-2b4b153d4035"}},"audit":{"status":"passed","reviewedAt":"2026-11-06T22:03:28.158Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/rakshit-s-trivedi-and-co-authors-argue-in-arxiv-for-cooperative-1bb3b466","formats":{"html":"/readings/rakshit-s-trivedi-and-co-authors-argue-in-arxiv-for-cooperative-1bb3b466","markdown":"/readings/rakshit-s-trivedi-and-co-authors-argue-in-arxiv-for-cooperative-1bb3b466.md","json":"/readings/rakshit-s-trivedi-and-co-authors-argue-in-arxiv-for-cooperative-1bb3b466.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-superintelligence","name":"Superintelligence","explanation":"A hypothetical AI far more capable than people across most intellectual tasks.","referenceUrl":"https://en.wikipedia.org/wiki/Superintelligence"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"},{"id":"google-deepmind","name":"Google DeepMind","kind":"organization","context":"Google DeepMind develops AI systems for general tasks and scientific research, including AlphaFold.","referenceUrls":["https://deepmind.google/blog/","https://deepmind.google/science/alphafold/"],"path":"/entities/google-deepmind","json":"/entities/google-deepmind.json","markdown":"/entities/google-deepmind.md"},{"id":"the-world","name":"The World","kind":"organization","context":"The World is a public radio news program publishing international reporting on politics, society, labor and technology.","referenceUrls":["https://theworld.org/"],"path":"/entities/the-world","json":"/entities/the-world.json","markdown":"/entities/the-world.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"1bb3b466765fb7db5fc6edc5c9c98c05eb26528ea7dc12aa0acfc344fef8dd66","revision":"4b4b3318-3306-65a0-12f8-eca53f06d12a","revisionKind":"publication-id","title":"Rakshit S Trivedi and co-authors argue in arXiv for cooperative AI design","url":"https://escapemeridian.com/readings/rakshit-s-trivedi-and-co-authors-argue-in-arxiv-for-cooperative-1bb3b466"},"source":{"type":"source","id":"https://deepmind.google/research/publications/231466/","revision":"c569e2f4d6d5c45f1790602b3465b1f9fa35af104ebb5df396d0bf3ae26d93f9","revisionKind":"sha256-source-text","title":"Solipsistic superintelligence is unlikely to be cooperative","url":"https://deepmind.google/research/publications/231466/"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"1bb3b466765fb7db5fc6edc5c9c98c05eb26528ea7dc12aa0acfc344fef8dd66","title":"Rakshit S Trivedi and co-authors argue in arXiv for cooperative AI design","argument":"The authors’ work was published on June 4, 2026. Rakshit S Trivedi, Natasha Jaques, Logan Cross, Alexander Sasha Vezhnevets and Joel Z Leibo argue that a superintelligence developed as a solipsistic task solver is unlikely to cooperate. They say the usual approach treats the world as a fixed source of feedback, but deploying AI systems changes the conditions those systems encounter. They call this mismatch the “self-undermining property of unilateral optimization.” In their account, cooperation is not simply another task for AI to solve: it is how multiple actors navigate their interdependence. The authors call for research that makes this interdependence central to AI design, including evaluations with adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of AI systems. Their argument shifts attention from capability alone to coexistence, and makes human agency part of the design question. The abstract does not specify how these proposals would be implemented or demonstrate that they would make AI systems cooperative.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on November 2, 2026; originally published June 4, 2026. The available evidence is the publication page’s abstract; it does not provide details of the proposed methods or evidence that they would produce cooperative systems.","speaker":{"name":"Google DeepMind","roleAtSource":"Publisher"},"source":{"title":"Solipsistic superintelligence is unlikely to be cooperative","publisher":"Google DeepMind","format":"Article","url":"https://deepmind.google/research/publications/231466/","publishedAt":"2026-06-04T00:00:00+00:00","dateBasis":"Date stated on the publisher’s page."},"evidence":[{"text":"The authors argue that superintelligence developed through a solipsistic approach to AI design is unlikely to be cooperative.","excerpt":"superintelligence, an extremely capable task solver, born out of such a solipsistic approach to AI design, is unlikely to be cooperative.","url":"https://deepmind.google/research/publications/231466/","relationship":"Source-stated"},{"text":"The authors say deploying AI systems changes the conditions they encounter, creating a gap between historical distributions and deployment context.","excerpt":"Deploying AI systems induces endogenous non-stationarity, resulting in a train-test-deploy gap where historical distributions diverge from the deployment context.","url":"https://deepmind.google/research/publications/231466/","relationship":"Source-stated"},{"text":"The authors define cooperation as an equilibrium-selection process through which multiple actors navigate interdependence.","excerpt":"the equilibrium-selection process through which multiple actors navigate their interdependence.","url":"https://deepmind.google/research/publications/231466/","relationship":"Source-stated"},{"text":"The authors call for a research paradigm that treats interdependence as a core design principle, with adaptive evaluations, institutions as design primitives, and human agency as a structural feature.","excerpt":"This entails building dynamic evaluation testbeds involving adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of the systems we build.","url":"https://deepmind.google/research/publications/231466/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://deepmind.google/research/publications/231466/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How would dynamic evaluations involving adaptive counterparties show whether an AI system can participate in cooperation?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Rakshit S Trivedi and co-authors argue in arXiv for cooperative AI design","summary":"The authors’ work was published on June 4, 2026. Rakshit S Trivedi, Natasha Jaques, Logan Cross, Alexander Sasha Vezhnevets and Joel Z Leibo argue that a superintelligence developed as a solipsistic task solver is unlikely to cooperate. They say the usual approach treats the world as a fixed source of feedback, but deploying AI systems changes the conditions those systems encounter. They call this mismatch the “self-undermining property of unilateral optimization.” In their account, cooperation is not simply another task for AI to solve: it is how multiple actors navigate their interdependence. The authors call for research that makes this interdependence central to AI design, including evaluations with adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of AI systems. Their argument shifts attention from capability alone to coexistence, and makes human agency part of the design question. The abstract does not specify how these proposals would be implemented or demonstrate that they would make AI systems cooperative.","question":"How would dynamic evaluations involving adaptive counterparties show whether an AI system can participate in cooperation?","claims":[{"text":"The authors argue that superintelligence developed through a solipsistic approach to AI design is unlikely to be cooperative.","excerpt":"superintelligence, an extremely capable task solver, born out of such a solipsistic approach to AI design, is unlikely to be cooperative.","support":"source-stated"},{"text":"The authors say deploying AI systems changes the conditions they encounter, creating a gap between historical distributions and deployment context.","excerpt":"Deploying AI systems induces endogenous non-stationarity, resulting in a train-test-deploy gap where historical distributions diverge from the deployment context.","support":"source-stated"},{"text":"The authors define cooperation as an equilibrium-selection process through which multiple actors navigate interdependence.","excerpt":"the equilibrium-selection process through which multiple actors navigate their interdependence.","support":"source-stated"},{"text":"The authors call for a research paradigm that treats interdependence as a core design principle, with adaptive evaluations, institutions as design primitives, and human agency as a structural feature.","excerpt":"This entails building dynamic evaluation testbeds involving adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of the systems we build.","support":"source-stated"}],"limitations":["Text as retrieved on November 2, 2026; originally published June 4, 2026.","The available evidence is the publication page’s abstract; it does not provide details of the proposed methods or evidence that they would produce cooperative systems."],"mediaId":null,"correctionReason":null,"id":"1bb3b466765fb7db5fc6edc5c9c98c05eb26528ea7dc12aa0acfc344fef8dd66","publicationId":"4b4b3318-3306-65a0-12f8-eca53f06d12a","version":1,"sourceUrl":"https://deepmind.google/research/publications/231466/","sourceTitle":"Solipsistic superintelligence is unlikely to be cooperative","publisher":"Google DeepMind","publishedAt":"2026-06-04T00:00:00+00:00","originalPublished":{"value":"2026-06-04T00:00:00+00:00","precision":"instant","basis":"backfill:article_published"},"firstObservedAt":"2026-11-02T23:25:57.408Z","createdAt":"2026-11-06T22:08:24.340Z","historical":true,"entityIds":["google-deepmind"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://deepmind.google/research/publications/231466/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"0f3b32058535781e641b658d37e1fffd910c50c7476afefb01280b8191795cdd","rubricVersion":"unrecorded","provider":"anthropic","model":"claude-opus-5-5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-11-02T23:25:54.324Z"},"provenance":{"textHash":"c569e2f4d6d5c45f1790602b3465b1f9fa35af104ebb5df396d0bf3ae26d93f9","observationId":"39b65c32f6dcc23587e7dda8301488021a8f60fc2afe14f12022615999f9cdb9","retrievedAt":"2026-11-02T23:27:21.125Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"d5c85510-db0c-a3cb-64fb-e22fbd019f00"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"cfb25d1b-e9b2-6a66-9b38-2b4b153d4035"}},"audit":{"status":"passed","reviewedAt":"2026-11-06T22:03:28.158Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"4b4b3318-3306-65a0-12f8-eca53f06d12a","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading names the five listed authors and credits the argument to them. The arXiv venue matches the page. It is a single publication abstract, not a listing.","excerpt":"Rakshit S Trivedi, Natasha Jaques, Logan Cross, Alexander Sasha Vezhnevets, Joel Z Leibo"},{"dimension":"summary","status":"pass","explanation":"The summary faithfully gives the thesis, the deployment mismatch, the authors' view of cooperation and their three proposals. It keeps the claims attributed and the framing of a shift from capability to coexistence. The term solipsistic is explained in context as treating the world as a fixed source of feedback, and superintelligence is defined on the page. Terms such as design primitives and adaptive counterparties remain unexplained but are not central. The question follows from the source. The limitation correctly says no implementation details or evidence are given.","excerpt":"AI's central challenge is shifting from capability to coexistence."},{"dimension":"date","status":"pass","explanation":"The June 4, 2026 date matches both the page dateline and evidence.originalPublished. The source gives the date at day precision. There is no hindsight framing. The evidence is a publisher page, so no archive disclosure is needed.","excerpt":"June 4, 2026"},{"dimension":"support","status":"pass","explanation":"Each claim is source-stated and backed by an exact excerpt. The summary implies no benefits, costs or outcomes beyond the abstract. It also states that the abstract does not demonstrate the proposals would work.","excerpt":"We refer to this as the self-undermining property of unilateral optimization."},{"dimension":"media","status":"pass","explanation":"No media was selected and no verified candidates were supplied, so nothing is misidentified or irrelevant.","excerpt":null}],"createdAt":"2026-11-06T22:03:28.158Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}