{"id":"4e39a3df2e11ce6caef15b6ace07f02bd6b0c6941cb3b85420a6b217dabd9a2a","title":"Dwarkesh Podcast hosts a debate on recursive self-improvement","argument":"The Dwarkesh Podcast transcript was published on September 11, 2026. In a conversation with Dwarkesh Patel, Beren Millidge, John Schulman and Charlie O’Neill debate how AI might automate AI research and whether that could lead to recursive self-improvement (RSI). They disagree about how likely rapid progress is, but return to a shared difficulty: making progress on a well-specified task may be easier than deciding what task or objective should come next. Millidge argues that an automated improvement loop would need to propose and optimize objectives repeatedly without going off course. Schulman says that defining what AI should do may remain a human role, even if AI can handle technical work. The speakers also discuss whether reinforcement learning in simulated environments transfers to complicated real-world work, and whether models can learn continuously from deployment. Millidge and O’Neill describe routes for using deployment data to improve later models, while Schulman points to companies’ incentives as a barrier to sharing that data. The conversation’s closing predictions about AI capabilities are the speakers’ forecasts, not reported outcomes. Whether AI can generalize well enough to set its own objectives and sustain an improvement loop without human involvement remains unresolved.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 25, 2026; originally published September 11, 2026. The source is a publisher transcript. The transcript presents a discussion among Patel, Millidge, Schulman and O’Neill; their views and forecasts are not independent demonstrations of the outcomes discussed.","speaker":{"name":"Dwarkesh Podcast","roleAtSource":"Publisher"},"source":{"title":"AI researchers debate how close we are to recursive self-improvement","publisher":"Dwarkesh Podcast","format":"Transcript","url":"https://www.dwarkesh.com/p/john-beren-charlie","publishedAt":"2026-09-11T16:28:11.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Millidge says a self-propelling automated loop would require AI to repeatedly propose and optimize objectives without going off course.","excerpt":"we need the AI to propose objectives, optimize them, figure that out, propose a new objective, and have this not go off the rails at any point for a long, long time.","url":"https://www.dwarkesh.com/p/john-beren-charlie","relationship":"Source-stated"},{"text":"Schulman says defining the objective and deciding what people want may be the human role that lasts longest.","excerpt":"the last job for humans, or the role for humans that’ll last the longest, is defining the objective and deciding what we actually want.","url":"https://www.dwarkesh.com/p/john-beren-charlie","relationship":"Source-stated"},{"text":"The speakers discuss possible improvements from deployment data, while Schulman identifies business incentives as a potential obstacle to sharing it.","excerpt":"whether we get a hive mind that learns from all of its deployment experience is in a big part about incentives, rather than being a technical question.","url":"https://www.dwarkesh.com/p/john-beren-charlie","relationship":"Source-stated"},{"text":"Millidge says it is unclear whether training across different tasks will generalize enough to remove humans from the loop.","excerpt":"Whether it generalizes enough that the loop can become self-sealing without humans being in the loop at all is unclear.","url":"https://www.dwarkesh.com/p/john-beren-charlie","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.dwarkesh.com/p/john-beren-charlie","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What evidence would show that AI can reliably propose useful new research objectives, rather than only make progress on objectives people have already specified?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Dwarkesh Podcast hosts a debate on recursive self-improvement","summary":"The Dwarkesh Podcast transcript was published on September 11, 2026. In a conversation with Dwarkesh Patel, Beren Millidge, John Schulman and Charlie O’Neill debate how AI might automate AI research and whether that could lead to recursive self-improvement (RSI). They disagree about how likely rapid progress is, but return to a shared difficulty: making progress on a well-specified task may be easier than deciding what task or objective should come next. Millidge argues that an automated improvement loop would need to propose and optimize objectives repeatedly without going off course. Schulman says that defining what AI should do may remain a human role, even if AI can handle technical work. The speakers also discuss whether reinforcement learning in simulated environments transfers to complicated real-world work, and whether models can learn continuously from deployment. Millidge and O’Neill describe routes for using deployment data to improve later models, while Schulman points to companies’ incentives as a barrier to sharing that data. The conversation’s closing predictions about AI capabilities are the speakers’ forecasts, not reported outcomes. Whether AI can generalize well enough to set its own objectives and sustain an improvement loop without human involvement remains unresolved.","question":"What evidence would show that AI can reliably propose useful new research objectives, rather than only make progress on objectives people have already specified?","claims":[{"text":"Millidge says a self-propelling automated loop would require AI to repeatedly propose and optimize objectives without going off course.","excerpt":"we need the AI to propose objectives, optimize them, figure that out, propose a new objective, and have this not go off the rails at any point for a long, long time.","support":"source-stated"},{"text":"Schulman says defining the objective and deciding what people want may be the human role that lasts longest.","excerpt":"the last job for humans, or the role for humans that’ll last the longest, is defining the objective and deciding what we actually want.","support":"source-stated"},{"text":"The speakers discuss possible improvements from deployment data, while Schulman identifies business incentives as a potential obstacle to sharing it.","excerpt":"whether we get a hive mind that learns from all of its deployment experience is in a big part about incentives, rather than being a technical question.","support":"source-stated"},{"text":"Millidge says it is unclear whether training across different tasks will generalize enough to remove humans from the loop.","excerpt":"Whether it generalizes enough that the loop can become self-sealing without humans being in the loop at all is unclear.","support":"source-stated"}],"limitations":["Text as retrieved on October 25, 2026; originally published September 11, 2026. The source is a publisher transcript.","The transcript presents a discussion among Patel, Millidge, Schulman and O’Neill; their views and forecasts are not independent demonstrations of the outcomes discussed."],"mediaId":null,"correctionReason":null,"id":"4e39a3df2e11ce6caef15b6ace07f02bd6b0c6941cb3b85420a6b217dabd9a2a","publicationId":"47dc5c9b-69f3-7a29-db06-480db5f6ee14","version":1,"sourceUrl":"https://www.dwarkesh.com/p/john-beren-charlie","sourceTitle":"AI researchers debate how close we are to recursive self-improvement","publisher":"Dwarkesh Podcast","publishedAt":"2026-09-11T16:28:11.000Z","originalPublished":{"value":"2026-09-11T16:28:11.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-09-23T05:24:05.590Z","createdAt":"2026-11-06T04:23:23.036Z","historical":true,"entityIds":["dwarkesh-podcast"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.dwarkesh.com/p/john-beren-charlie","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"d4d73fa0624ead7c2bda738fe04e513c04acd508ef537cdf464b3dbb00e66221","rubricVersion":"unrecorded","provider":"unrecorded","model":"anthropic/claude-opus-5.5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-25T05:54:03.486Z"},"provenance":{"textHash":"e2edf7df0e230d21d17a75db0e4e3b0f4d39700c77c93edff3082fb6d7730917","observationId":"2104a6275e7f717ea787a7d83ddd3deb268509881c5163240f693eba3bc76afb","retrievedAt":"2026-10-25T05:09:42.682Z","kind":"publisher-transcript","preparer":{"model":"openai/gpt-6-luna","runId":"3f702fc4-da30-c3b0-d228-af5987b947b7"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"563c5e98-b75f-d0f1-a3ae-7bead2ffde8f"}},"audit":{"status":"passed","reviewedAt":"2026-11-06T04:18:25.926Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/dwarkesh-podcast-hosts-a-debate-on-recursive-self-improvement-4e39a3df","formats":{"html":"/readings/dwarkesh-podcast-hosts-a-debate-on-recursive-self-improvement-4e39a3df","markdown":"/readings/dwarkesh-podcast-hosts-a-debate-on-recursive-self-improvement-4e39a3df.md","json":"/readings/dwarkesh-podcast-hosts-a-debate-on-recursive-self-improvement-4e39a3df.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-recursive-self-improvement","name":"Recursive self-improvement","explanation":"An AI system improving its own capabilities, such as its code, training methods or reasoning, then using the gains to improve further. Each cycle could hypothetically produce a more capable system.","referenceUrl":"https://hai.stanford.edu/what-is-recursive-self-improvement"},{"id":"term-reinforcement-learning","name":"Reinforcement learning","explanation":"Training by trial and error, in which a system is rewarded for the outcomes its designers want.","referenceUrl":"https://developers.google.com/machine-learning/glossary#reinforcement-learning-rl"}],"entities":[{"id":"dwarkesh-podcast","name":"Dwarkesh Podcast","kind":"show","context":"Dwarkesh Podcast features long conversations with researchers, builders and thinkers about technology, history and the future.","referenceUrls":["https://www.dwarkesh.com/"],"path":"/entities/dwarkesh-podcast","json":"/entities/dwarkesh-podcast.json","markdown":"/entities/dwarkesh-podcast.md"},{"id":"dwarkesh-patel","name":"Dwarkesh Patel","kind":"person","context":"Dwarkesh Patel hosts the Dwarkesh Podcast, asking researchers and other guests to explain and defend their ideas.","referenceUrls":["https://www.dwarkesh.com/about","https://www.dwarkesh.com/"],"path":"/entities/dwarkesh-patel","json":"/entities/dwarkesh-patel.json","markdown":"/entities/dwarkesh-patel.md"},{"id":"beren-millidge","name":"Beren Millidge","kind":"person","context":"Beren Millidge is introduced as Zyphra's chief technology officer developing open models in the September 2026 Dwarkesh panel on AI research.","referenceUrls":["https://www.dwarkesh.com/p/john-beren-charlie"],"path":"/entities/beren-millidge","json":"/entities/beren-millidge.json","markdown":"/entities/beren-millidge.md"},{"id":"john-schulman","name":"John Schulman","kind":"person","context":"John Schulman is introduced as Thinking Machines' chief scientist in the September 2026 Dwarkesh panel on AI research.","referenceUrls":["https://www.dwarkesh.com/p/john-beren-charlie"],"path":"/entities/john-schulman","json":"/entities/john-schulman.json","markdown":"/entities/john-schulman.md"},{"id":"charlie-oneill","name":"Charlie O’Neill","kind":"person","context":"Charlie O’Neill is introduced as Baseten's head of model training in the September 2026 Dwarkesh panel on AI research.","referenceUrls":["https://www.dwarkesh.com/p/john-beren-charlie"],"path":"/entities/charlie-oneill","json":"/entities/charlie-oneill.json","markdown":"/entities/charlie-oneill.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"4e39a3df2e11ce6caef15b6ace07f02bd6b0c6941cb3b85420a6b217dabd9a2a","revision":"47dc5c9b-69f3-7a29-db06-480db5f6ee14","revisionKind":"publication-id","title":"Dwarkesh Podcast hosts a debate on recursive self-improvement","url":"https://escapemeridian.com/readings/dwarkesh-podcast-hosts-a-debate-on-recursive-self-improvement-4e39a3df"},"source":{"type":"source","id":"https://www.dwarkesh.com/p/john-beren-charlie","revision":"e2edf7df0e230d21d17a75db0e4e3b0f4d39700c77c93edff3082fb6d7730917","revisionKind":"sha256-source-text","title":"AI researchers debate how close we are to recursive self-improvement","url":"https://www.dwarkesh.com/p/john-beren-charlie"},"status":"available","items":[{"id":"reading-citation:millidge-learning-beyond-demonstrations:a35c59c103a322444b14b9ef1c5a3df946bf199372f2745ce1d32ba98fbca648:evidence.0.url","from":{"type":"reading","id":"millidge-learning-beyond-demonstrations","revision":"a35c59c103a322444b14b9ef1c5a3df946bf199372f2745ce1d32ba98fbca648","revisionKind":"sha256-record","title":"Beren Millidge: a clear target lets a model try what people have not demonstrated","url":"https://escapemeridian.com/readings/millidge-learning-beyond-demonstrations"},"to":{"type":"source","id":"https://www.dwarkesh.com/p/john-beren-charlie","revision":null,"revisionKind":null,"title":"AI researchers debate how close we are to recursive self-improvement","url":"https://www.dwarkesh.com/p/john-beren-charlie"},"relationship":"references","provenance":{"kind":"recorded-citation","recordId":"a35c59c103a322444b14b9ef1c5a3df946bf199372f2745ce1d32ba98fbca648","field":"evidence.0.url","recordedAt":null,"author":{"id":null,"name":"Escape Meridian","kind":"editorial"},"citedUrl":"https://www.dwarkesh.com/p/john-beren-charlie","assertedRelationship":"supports"}},{"id":"reading-citation:millidge-learning-beyond-demonstrations:a35c59c103a322444b14b9ef1c5a3df946bf199372f2745ce1d32ba98fbca648:source.url","from":{"type":"reading","id":"millidge-learning-beyond-demonstrations","revision":"a35c59c103a322444b14b9ef1c5a3df946bf199372f2745ce1d32ba98fbca648","revisionKind":"sha256-record","title":"Beren Millidge: a clear target lets a model try what people have not demonstrated","url":"https://escapemeridian.com/readings/millidge-learning-beyond-demonstrations"},"to":{"type":"source","id":"https://www.dwarkesh.com/p/john-beren-charlie","revision":null,"revisionKind":null,"title":"AI researchers debate how close we are to recursive self-improvement","url":"https://www.dwarkesh.com/p/john-beren-charlie"},"relationship":"references","provenance":{"kind":"recorded-citation","recordId":"a35c59c103a322444b14b9ef1c5a3df946bf199372f2745ce1d32ba98fbca648","field":"source.url","recordedAt":null,"author":{"id":null,"name":"Escape Meridian","kind":"editorial"},"citedUrl":"https://www.dwarkesh.com/p/john-beren-charlie"}},{"id":"reading-citation:oneill-changing-work-context:bc58185488bb7c8e1ddb2fbb3018f337ff322386170a8cdd041a161e1a8a6763:evidence.0.url","from":{"type":"reading","id":"oneill-changing-work-context","revision":"bc58185488bb7c8e1ddb2fbb3018f337ff322386170a8cdd041a161e1a8a6763","revisionKind":"sha256-record","title":"Charlie O’Neill: useful knowledge can become a wrong assumption as work changes","url":"https://escapemeridian.com/readings/oneill-changing-work-context"},"to":{"type":"source","id":"https://www.dwarkesh.com/p/john-beren-charlie","revision":null,"revisionKind":null,"title":"AI researchers debate how close we are to recursive self-improvement","url":"https://www.dwarkesh.com/p/john-beren-charlie"},"relationship":"references","provenance":{"kind":"recorded-citation","recordId":"bc58185488bb7c8e1ddb2fbb3018f337ff322386170a8cdd041a161e1a8a6763","field":"evidence.0.url","recordedAt":null,"author":{"id":null,"name":"Escape Meridian","kind":"editorial"},"citedUrl":"https://www.dwarkesh.com/p/john-beren-charlie","assertedRelationship":"supports"}},{"id":"reading-citation:oneill-changing-work-context:bc58185488bb7c8e1ddb2fbb3018f337ff322386170a8cdd041a161e1a8a6763:source.url","from":{"type":"reading","id":"oneill-changing-work-context","revision":"bc58185488bb7c8e1ddb2fbb3018f337ff322386170a8cdd041a161e1a8a6763","revisionKind":"sha256-record","title":"Charlie O’Neill: useful knowledge can become a wrong assumption as work changes","url":"https://escapemeridian.com/readings/oneill-changing-work-context"},"to":{"type":"source","id":"https://www.dwarkesh.com/p/john-beren-charlie","revision":null,"revisionKind":null,"title":"AI researchers debate how close we are to recursive self-improvement","url":"https://www.dwarkesh.com/p/john-beren-charlie"},"relationship":"references","provenance":{"kind":"recorded-citation","recordId":"bc58185488bb7c8e1ddb2fbb3018f337ff322386170a8cdd041a161e1a8a6763","field":"source.url","recordedAt":null,"author":{"id":null,"name":"Escape Meridian","kind":"editorial"},"citedUrl":"https://www.dwarkesh.com/p/john-beren-charlie"}},{"id":"reading-citation:schulman-choosing-objectives:5de0b7d92e886f3f97e0ac5fe634776116850c9030b964b3a2ee1656f371d027:evidence.0.url","from":{"type":"reading","id":"schulman-choosing-objectives","revision":"5de0b7d92e886f3f97e0ac5fe634776116850c9030b964b3a2ee1656f371d027","revisionKind":"sha256-record","title":"John Schulman: someone still has to decide what a good result means","url":"https://escapemeridian.com/readings/schulman-choosing-objectives"},"to":{"type":"source","id":"https://www.dwarkesh.com/p/john-beren-charlie","revision":null,"revisionKind":null,"title":"AI researchers debate how close we are to recursive self-improvement","url":"https://www.dwarkesh.com/p/john-beren-charlie"},"relationship":"references","provenance":{"kind":"recorded-citation","recordId":"5de0b7d92e886f3f97e0ac5fe634776116850c9030b964b3a2ee1656f371d027","field":"evidence.0.url","recordedAt":null,"author":{"id":null,"name":"Escape Meridian","kind":"editorial"},"citedUrl":"https://www.dwarkesh.com/p/john-beren-charlie","assertedRelationship":"supports"}},{"id":"reading-citation:schulman-choosing-objectives:5de0b7d92e886f3f97e0ac5fe634776116850c9030b964b3a2ee1656f371d027:source.url","from":{"type":"reading","id":"schulman-choosing-objectives","revision":"5de0b7d92e886f3f97e0ac5fe634776116850c9030b964b3a2ee1656f371d027","revisionKind":"sha256-record","title":"John Schulman: someone still has to decide what a good result means","url":"https://escapemeridian.com/readings/schulman-choosing-objectives"},"to":{"type":"source","id":"https://www.dwarkesh.com/p/john-beren-charlie","revision":null,"revisionKind":null,"title":"AI researchers debate how close we are to recursive self-improvement","url":"https://www.dwarkesh.com/p/john-beren-charlie"},"relationship":"references","provenance":{"kind":"recorded-citation","recordId":"5de0b7d92e886f3f97e0ac5fe634776116850c9030b964b3a2ee1656f371d027","field":"source.url","recordedAt":null,"author":{"id":null,"name":"Escape Meridian","kind":"editorial"},"citedUrl":"https://www.dwarkesh.com/p/john-beren-charlie"}}],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"4e39a3df2e11ce6caef15b6ace07f02bd6b0c6941cb3b85420a6b217dabd9a2a","title":"Dwarkesh Podcast hosts a debate on recursive self-improvement","argument":"The Dwarkesh Podcast transcript was published on September 11, 2026. In a conversation with Dwarkesh Patel, Beren Millidge, John Schulman and Charlie O’Neill debate how AI might automate AI research and whether that could lead to recursive self-improvement (RSI). They disagree about how likely rapid progress is, but return to a shared difficulty: making progress on a well-specified task may be easier than deciding what task or objective should come next. Millidge argues that an automated improvement loop would need to propose and optimize objectives repeatedly without going off course. Schulman says that defining what AI should do may remain a human role, even if AI can handle technical work. The speakers also discuss whether reinforcement learning in simulated environments transfers to complicated real-world work, and whether models can learn continuously from deployment. Millidge and O’Neill describe routes for using deployment data to improve later models, while Schulman points to companies’ incentives as a barrier to sharing that data. The conversation’s closing predictions about AI capabilities are the speakers’ forecasts, not reported outcomes. Whether AI can generalize well enough to set its own objectives and sustain an improvement loop without human involvement remains unresolved.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 25, 2026; originally published September 11, 2026. The source is a publisher transcript. The transcript presents a discussion among Patel, Millidge, Schulman and O’Neill; their views and forecasts are not independent demonstrations of the outcomes discussed.","speaker":{"name":"Dwarkesh Podcast","roleAtSource":"Publisher"},"source":{"title":"AI researchers debate how close we are to recursive self-improvement","publisher":"Dwarkesh Podcast","format":"Transcript","url":"https://www.dwarkesh.com/p/john-beren-charlie","publishedAt":"2026-09-11T16:28:11.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Millidge says a self-propelling automated loop would require AI to repeatedly propose and optimize objectives without going off course.","excerpt":"we need the AI to propose objectives, optimize them, figure that out, propose a new objective, and have this not go off the rails at any point for a long, long time.","url":"https://www.dwarkesh.com/p/john-beren-charlie","relationship":"Source-stated"},{"text":"Schulman says defining the objective and deciding what people want may be the human role that lasts longest.","excerpt":"the last job for humans, or the role for humans that’ll last the longest, is defining the objective and deciding what we actually want.","url":"https://www.dwarkesh.com/p/john-beren-charlie","relationship":"Source-stated"},{"text":"The speakers discuss possible improvements from deployment data, while Schulman identifies business incentives as a potential obstacle to sharing it.","excerpt":"whether we get a hive mind that learns from all of its deployment experience is in a big part about incentives, rather than being a technical question.","url":"https://www.dwarkesh.com/p/john-beren-charlie","relationship":"Source-stated"},{"text":"Millidge says it is unclear whether training across different tasks will generalize enough to remove humans from the loop.","excerpt":"Whether it generalizes enough that the loop can become self-sealing without humans being in the loop at all is unclear.","url":"https://www.dwarkesh.com/p/john-beren-charlie","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.dwarkesh.com/p/john-beren-charlie","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What evidence would show that AI can reliably propose useful new research objectives, rather than only make progress on objectives people have already specified?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Dwarkesh Podcast hosts a debate on recursive self-improvement","summary":"The Dwarkesh Podcast transcript was published on September 11, 2026. In a conversation with Dwarkesh Patel, Beren Millidge, John Schulman and Charlie O’Neill debate how AI might automate AI research and whether that could lead to recursive self-improvement (RSI). They disagree about how likely rapid progress is, but return to a shared difficulty: making progress on a well-specified task may be easier than deciding what task or objective should come next. Millidge argues that an automated improvement loop would need to propose and optimize objectives repeatedly without going off course. Schulman says that defining what AI should do may remain a human role, even if AI can handle technical work. The speakers also discuss whether reinforcement learning in simulated environments transfers to complicated real-world work, and whether models can learn continuously from deployment. Millidge and O’Neill describe routes for using deployment data to improve later models, while Schulman points to companies’ incentives as a barrier to sharing that data. The conversation’s closing predictions about AI capabilities are the speakers’ forecasts, not reported outcomes. Whether AI can generalize well enough to set its own objectives and sustain an improvement loop without human involvement remains unresolved.","question":"What evidence would show that AI can reliably propose useful new research objectives, rather than only make progress on objectives people have already specified?","claims":[{"text":"Millidge says a self-propelling automated loop would require AI to repeatedly propose and optimize objectives without going off course.","excerpt":"we need the AI to propose objectives, optimize them, figure that out, propose a new objective, and have this not go off the rails at any point for a long, long time.","support":"source-stated"},{"text":"Schulman says defining the objective and deciding what people want may be the human role that lasts longest.","excerpt":"the last job for humans, or the role for humans that’ll last the longest, is defining the objective and deciding what we actually want.","support":"source-stated"},{"text":"The speakers discuss possible improvements from deployment data, while Schulman identifies business incentives as a potential obstacle to sharing it.","excerpt":"whether we get a hive mind that learns from all of its deployment experience is in a big part about incentives, rather than being a technical question.","support":"source-stated"},{"text":"Millidge says it is unclear whether training across different tasks will generalize enough to remove humans from the loop.","excerpt":"Whether it generalizes enough that the loop can become self-sealing without humans being in the loop at all is unclear.","support":"source-stated"}],"limitations":["Text as retrieved on October 25, 2026; originally published September 11, 2026. The source is a publisher transcript.","The transcript presents a discussion among Patel, Millidge, Schulman and O’Neill; their views and forecasts are not independent demonstrations of the outcomes discussed."],"mediaId":null,"correctionReason":null,"id":"4e39a3df2e11ce6caef15b6ace07f02bd6b0c6941cb3b85420a6b217dabd9a2a","publicationId":"47dc5c9b-69f3-7a29-db06-480db5f6ee14","version":1,"sourceUrl":"https://www.dwarkesh.com/p/john-beren-charlie","sourceTitle":"AI researchers debate how close we are to recursive self-improvement","publisher":"Dwarkesh Podcast","publishedAt":"2026-09-11T16:28:11.000Z","originalPublished":{"value":"2026-09-11T16:28:11.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-09-23T05:24:05.590Z","createdAt":"2026-11-06T04:23:23.036Z","historical":true,"entityIds":["dwarkesh-podcast"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.dwarkesh.com/p/john-beren-charlie","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"d4d73fa0624ead7c2bda738fe04e513c04acd508ef537cdf464b3dbb00e66221","rubricVersion":"unrecorded","provider":"unrecorded","model":"anthropic/claude-opus-5.5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-25T05:54:03.486Z"},"provenance":{"textHash":"e2edf7df0e230d21d17a75db0e4e3b0f4d39700c77c93edff3082fb6d7730917","observationId":"2104a6275e7f717ea787a7d83ddd3deb268509881c5163240f693eba3bc76afb","retrievedAt":"2026-10-25T05:09:42.682Z","kind":"publisher-transcript","preparer":{"model":"openai/gpt-6-luna","runId":"3f702fc4-da30-c3b0-d228-af5987b947b7"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"563c5e98-b75f-d0f1-a3ae-7bead2ffde8f"}},"audit":{"status":"passed","reviewedAt":"2026-11-06T04:18:25.926Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"47dc5c9b-69f3-7a29-db06-480db5f6ee14","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"This is a single publisher transcript of one Dwarkesh Podcast episode. The reading correctly attributes views to Dwarkesh Patel, Beren Millidge, John Schulman and Charlie O’Neill. Each claim is tied to the right speaker and stays an attributed assertion rather than an established fact.","excerpt":"I’m joined by Beren Millidge, who is the CTO of Zyphra, which is developing open source models. John Schulman is the chief scientist at Thinking Machines"},{"dimension":"summary","status":"pass","explanation":"The summary faithfully states the central debate: automating AI research, and whether choosing objectives is harder than optimizing them. It also covers sim-to-real transfer, continual learning from deployment, and Schulman’s point about incentives. The title and question follow from the evidence. Recursive self-improvement and reinforcement learning are defined beside the reading, so a non-specialist can follow the actors and the stakes.","excerpt":"Just thinking doesn’t necessarily buy you the right objective in the first place."},{"dimension":"date","status":"pass","explanation":"The reading’s September 11, 2026 matches the UTC day of evidence.originalPublished. The kind is publisher-transcript, not archive-copy, so no archive disclosure is required. modifiedAt equals the publication time. The later Recent Episodes sidebar entries are neither used nor relied on. The closing timeline predictions are explicitly framed as the speakers’ forecasts, not outcomes.","excerpt":null},{"dimension":"support","status":"pass","explanation":"All four claim excerpts are exact substrings of the transcript and support their claims as stated. The summary’s broader statements about disagreement over pace, sim-to-real transfer and deployment-data routes are grounded in the discussion. No implied benefits, costs or outcomes go beyond the source.","excerpt":"Whether it generalizes enough that the loop can become self-sealing without humans being in the loop at all is unclear."},{"dimension":"media","status":"pass","explanation":"No media is selected, and no verified media candidates were supplied. Leaving media null is appropriate, and no link was invented.","excerpt":null}],"createdAt":"2026-11-06T04:18:25.926Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}