{"id":"schulman-choosing-objectives","title":"Someone still has to decide what a good result means","topicIds":["intelligence-and-discovery","work-and-agency"],"domain":"work","speaker":{"name":"John Schulman","roleAtSource":"Thinking Machines chief scientist, introduced in the panel","expertise":"AI researcher and former OpenAI co-founder, as introduced in the episode.","claimBoundary":"Research experience informs the argument; transfer to a particular workplace remains a test.","declaredInterests":[{"text":"Leads research at Thinking Machines.","url":"https://www.dwarkesh.com/p/john-beren-charlie"}],"bioSources":[{"title":"Panel introduction","url":"https://www.dwarkesh.com/p/john-beren-charlie"}]},"source":{"title":"AI researchers debate how close we are to recursive self-improvement","publisher":"Dwarkesh Podcast","format":"Panel transcript","url":"https://www.dwarkesh.com/p/john-beren-charlie","publishedAt":"2026-09-11","dateBasis":"Publication date displayed on the official episode page.","anchor":{"label":"00:00 chapter — defining the objective, late in this chapter","url":"https://www.dwarkesh.com/p/john-beren-charlie?t=0","timestampSeconds":0},"accessNote":"Publisher transcript read on 23 September 2026, New York time. Timestamp marks a chapter, not an exact quote. Audio/video not independently listened to."},"argument":"Schulman separates carrying out experiments from deciding what to seek. Useful research also involves judgment, realistic tasks and objectives that reflect what people actually want.","claimType":"interpretation","evidence":[{"text":"Schulman identifies deciding the objective as a lasting human role; later he warns that a signal such as accepting an edit may reward behavior other than useful work.","url":"https://www.dwarkesh.com/p/john-beren-charlie","kind":"interpretive","relationship":"supports"},{"text":"Cursor reports that its own reward design induced broken tool calls and excessive clarification before it corrected those incentives.","url":"https://cursor.com/blog/real-time-rl-for-composer","kind":"empirical","relationship":"context"}],"assumptions":["Editorial reconstruction: the chosen success measure captures the outcome people care about."],"whatWouldChangeView":{"attribution":"Editorial test, not a stated concession","text":"Give an assistant a real decision with competing aims. Assess the usefulness of its proposed investigation and final result, including costs the original metric missed."},"limits":"The panel itself discusses Composer. The connection to the specific failures in Cursor’s report is editorial; those failures are different from Schulman’s example of an edit-acceptance reward.","discussionQuestion":"What could an assistant improve on a scoreboard while making the real thing worse, at work, school or home?","relatedSignalIds":["research-panel-choosing-next-question","openai-research-activity-and-progress"],"editorialComparisons":[{"positionId":"mollick-retain-the-learning","relationship":"Editorial comparison; no direct exchange established","question":"Who retains the experience needed to recognize a misleadingly good result?"}],"sourceMedia":{"status":"found","sourceUrl":"https://www.dwarkesh.com/p/john-beren-charlie","checkedAt":"2026-10-15T20:07:33.560800Z","items":[{"type":"video","url":"https://www.youtube.com/watch?v=PrSf7IOYu-I","sourceUrl":"https://www.dwarkesh.com/p/john-beren-charlie","attribution":"Dwarkesh Podcast","basis":"publisher-embed","availability":"verified","verifiedAt":"2026-10-15T20:07:33.560800Z","label":"Watch the conversation","preview":{"url":"https://i.ytimg.com/vi/PrSf7IOYu-I/hqdefault.jpg","alt":"Episode thumbnail: AI researchers debate how close we are to recursive self-improvement","credit":"Dwarkesh Patel / YouTube"}}]},"canonicalUrl":"https://escapemeridian.com/readings/schulman-choosing-objectives","formats":{"html":"/readings/schulman-choosing-objectives","markdown":"/readings/schulman-choosing-objectives.md","json":"/readings/schulman-choosing-objectives.json"},"attribution":"Headline and sourced paraphrase by Escape Meridian. Inclusion is not endorsement.","readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[],"entities":[{"id":"john-schulman","name":"John Schulman","kind":"person","context":"John Schulman is introduced as Thinking Machines' chief scientist in the September 2026 Dwarkesh panel on AI research.","referenceUrls":["https://www.dwarkesh.com/p/john-beren-charlie"],"path":"/entities/john-schulman","json":"/entities/john-schulman.json","markdown":"/entities/john-schulman.md"},{"id":"dwarkesh-podcast","name":"Dwarkesh Podcast","kind":"show","context":"Dwarkesh Podcast features long conversations with researchers, builders and thinkers about technology, history and the future.","referenceUrls":["https://www.dwarkesh.com/"],"path":"/entities/dwarkesh-podcast","json":"/entities/dwarkesh-podcast.json","markdown":"/entities/dwarkesh-podcast.md"},{"id":"cursor","name":"Cursor","kind":"organization","context":"Cursor makes an AI coding environment and publishes evaluations of agents working on software tasks.","referenceUrls":["https://cursor.com/","https://cursor.com/evals","https://cursor.com/blog/real-time-rl-for-composer"],"path":"/entities/cursor","json":"/entities/cursor.json","markdown":"/entities/cursor.md"}]},"withdrawn":false,"revisions":[],"audits":[],"correctionUrl":"https://escapemeridian.com/corrections/"}