{"id":"6e34d3bc3150f702e2b7c1ae66bbe37a47c427bd7be0c9e31d36527c4bd6e5cf","title":"Sam Altman describes OpenAI’s Dota agents trained through self-play","argument":"Sam Altman’s post, published June 25, 2018, describes OpenAI training a team of agents to play Dota using PPO, a general reinforcement learning algorithm. Altman says the agents beat semi-pros and outperformed agents trained two weeks earlier, without training on human-played games. He says the team designed the reward functions, while the agents learned by playing against themselves.\n\nAltman argues that the result matters because it suggests deep reinforcement learning can tackle difficult problems when there is enough computing scale and a suitable simulated environment. He presents applying the approach to other problems as a hope, and describes environments increasingly like the real world as a possibility—not a demonstrated outcome. He also says many problems are too complex for hand-coded solutions and expects this approach to become a large branch of machine learning and an important step toward general intelligence.\n\nThe post does not provide details here about the evaluation setup or evidence from tasks beyond Dota. What conditions would let this self-play approach transfer from a game to other environments?","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on 2026-10-23; originally published 2018-06-25. The post does not provide details here about the evaluation setup or evidence of performance beyond the reported Dota comparisons. The possible use of this approach in other problems and environments is presented as a hope and an imagined application, not as a reported result.","speaker":{"name":"Sam Altman","roleAtSource":"Publisher"},"source":{"title":"Reinforcement Learning Progress","publisher":"Sam Altman","format":"Article","url":"https://blog.samaltman.com/reinforcement-learning-progress","publishedAt":"2018-06-25T16:10:37Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Altman says OpenAI used PPO to train a team of agents to play Dota and beat semi-pros.","excerpt":"We used PPO (Proximal Policy Optimization), a general reinforcement learning algorithm invented by OpenAI, to train a team of 5 agents to play Dota and beat semi-pros.","url":"https://blog.samaltman.com/reinforcement-learning-progress","relationship":"Source-stated"},{"text":"Altman says the agents outperformed agents trained two weeks earlier, with a win rate of 90–95%, without training on human-played games; the team designed reward functions and the algorithm learned through self-play.","excerpt":"The agents we train consistently outperform two-week old agents with a win rate of 90-95%. We did this without training on human-played games—we did design the reward functions, of course, but the algorithm figured out how to play by training against itself.","url":"https://blog.samaltman.com/reinforcement-learning-progress","relationship":"Source-stated"},{"text":"Altman argues that deep reinforcement learning can solve difficult problems when enough computing scale and a suitable simulated environment are available.","excerpt":"This is a big deal because it shows that deep reinforcement learning can solve extremely hard problems whenever you can throw enough computing scale and a really good simulated environment that captures the problem you’re solving.","url":"https://blog.samaltman.com/reinforcement-learning-progress","relationship":"Source-stated"},{"text":"Altman says he expects reinforcement learning to become a large branch of machine learning and an important step toward general intelligence.","excerpt":"I expect this to be a large branch of machine learning, and an important step on the road towards general intelligence.","url":"https://blog.samaltman.com/reinforcement-learning-progress","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://blog.samaltman.com/reinforcement-learning-progress","checkedAt":null,"items":[]},"assumptions":[],"whatWouldChangeView":{"attribution":"Open question","text":"What conditions would let this self-play approach transfer from a game to other environments?"},"editorialComparisons":[],"live":{"title":"Sam Altman describes OpenAI’s Dota agents trained through self-play","summary":"Sam Altman’s post, published June 25, 2018, describes OpenAI training a team of agents to play Dota using PPO, a general reinforcement learning algorithm. Altman says the agents beat semi-pros and outperformed agents trained two weeks earlier, without training on human-played games. He says the team designed the reward functions, while the agents learned by playing against themselves.\n\nAltman argues that the result matters because it suggests deep reinforcement learning can tackle difficult problems when there is enough computing scale and a suitable simulated environment. He presents applying the approach to other problems as a hope, and describes environments increasingly like the real world as a possibility—not a demonstrated outcome. He also says many problems are too complex for hand-coded solutions and expects this approach to become a large branch of machine learning and an important step toward general intelligence.\n\nThe post does not provide details here about the evaluation setup or evidence from tasks beyond Dota. What conditions would let this self-play approach transfer from a game to other environments?","question":"What conditions would let this self-play approach transfer from a game to other environments?","claims":[{"text":"Altman says OpenAI used PPO to train a team of agents to play Dota and beat semi-pros.","excerpt":"We used PPO (Proximal Policy Optimization), a general reinforcement learning algorithm invented by OpenAI, to train a team of 5 agents to play Dota and beat semi-pros.","support":"source-stated"},{"text":"Altman says the agents outperformed agents trained two weeks earlier, with a win rate of 90–95%, without training on human-played games; the team designed reward functions and the algorithm learned through self-play.","excerpt":"The agents we train consistently outperform two-week old agents with a win rate of 90-95%. We did this without training on human-played games—we did design the reward functions, of course, but the algorithm figured out how to play by training against itself.","support":"source-stated"},{"text":"Altman argues that deep reinforcement learning can solve difficult problems when enough computing scale and a suitable simulated environment are available.","excerpt":"This is a big deal because it shows that deep reinforcement learning can solve extremely hard problems whenever you can throw enough computing scale and a really good simulated environment that captures the problem you’re solving.","support":"source-stated"},{"text":"Altman says he expects reinforcement learning to become a large branch of machine learning and an important step toward general intelligence.","excerpt":"I expect this to be a large branch of machine learning, and an important step on the road towards general intelligence.","support":"source-stated"}],"limitations":["Text as retrieved on 2026-10-23; originally published 2018-06-25.","The post does not provide details here about the evaluation setup or evidence of performance beyond the reported Dota comparisons.","The possible use of this approach in other problems and environments is presented as a hope and an imagined application, not as a reported result."],"mediaId":null,"correctionReason":null,"id":"6e34d3bc3150f702e2b7c1ae66bbe37a47c427bd7be0c9e31d36527c4bd6e5cf","publicationId":"d8f03a50-6ea2-19ba-8f91-94eb5ed2b975","version":1,"sourceUrl":"https://blog.samaltman.com/reinforcement-learning-progress","sourceTitle":"Reinforcement Learning Progress","publisher":"Sam Altman","publishedAt":"2018-06-25T16:10:37Z","originalPublished":{"value":"2018-06-25T16:10:37Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T03:05:36.110Z","createdAt":"2026-10-29T00:28:08.105Z","historical":true,"entityIds":["sam-altman"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://blog.samaltman.com/reinforcement-learning-progress","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"972e28fdfb1f07857088f7dba03aa6760c409cda7eb9461f1fd207b7bcd85c8a","rubricVersion":"unrecorded","provider":"vercel","model":"typesafe-ai/jev","reasonCodes":["bridge_admitted_by_model"],"confidence":0.58,"decidedAt":"2026-10-23T21:33:45.424Z"},"provenance":{"textHash":"ecf1badfdd441266218d0a4c066852cb19911761645d8cf4421de09d0daf06ab","observationId":"2a774389f8ffe69c997516f03e610fcc40213638845e0f27b84197a5128cb523","retrievedAt":"2026-10-23T21:35:35.120Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"7e6929a1-1836-3df1-9301-9704e6c5b45e"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"5081e745-3223-b298-e274-b9db9976e871"}},"audit":{"status":"passed","reviewedAt":"2026-10-29T00:27:49.372Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/sam-altman-describes-openai-s-dota-agents-trained-through-self-6e34d3bc","formats":{"html":"/readings/sam-altman-describes-openai-s-dota-agents-trained-through-self-6e34d3bc","markdown":"/readings/sam-altman-describes-openai-s-dota-agents-trained-through-self-6e34d3bc.md","json":"/readings/sam-altman-describes-openai-s-dota-agents-trained-through-self-6e34d3bc.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-reinforcement-learning","name":"Reinforcement learning","explanation":"Training by trial and error, in which a system is rewarded for the outcomes its designers want.","referenceUrl":"https://developers.google.com/machine-learning/glossary#reinforcement-learning-rl"},{"id":"term-machine-learning","name":"Machine learning","explanation":"Methods that let computers learn patterns from examples instead of following rules written by hand.","referenceUrl":"https://developers.google.com/machine-learning/glossary#machine-learning"}],"entities":[{"id":"sam-altman","name":"Sam Altman","kind":"person","context":"Sam Altman publishes essays about AI and technology on his personal blog, separate from OpenAI announcements.","referenceUrls":["https://blog.samaltman.com/","https://openai.com/our-structure/","https://blog.samaltman.com/posts.atom"],"path":"/entities/sam-altman","json":"/entities/sam-altman.json","markdown":"/entities/sam-altman.md"},{"id":"openai","name":"OpenAI","kind":"organization","context":"OpenAI develops AI models and products and publishes research, release announcements and evaluations.","referenceUrls":["https://openai.com/news/"],"path":"/entities/openai","json":"/entities/openai.json","markdown":"/entities/openai.md"}]},"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"6e34d3bc3150f702e2b7c1ae66bbe37a47c427bd7be0c9e31d36527c4bd6e5cf","revision":"d8f03a50-6ea2-19ba-8f91-94eb5ed2b975","revisionKind":"publication-id","title":"Sam Altman describes OpenAI’s Dota agents trained through self-play","url":"https://escapemeridian.com/readings/sam-altman-describes-openai-s-dota-agents-trained-through-self-6e34d3bc"},"source":{"type":"source","id":"https://blog.samaltman.com/reinforcement-learning-progress","revision":"ecf1badfdd441266218d0a4c066852cb19911761645d8cf4421de09d0daf06ab","revisionKind":"sha256-source-text","title":"Reinforcement Learning Progress","url":"https://blog.samaltman.com/reinforcement-learning-progress"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"6e34d3bc3150f702e2b7c1ae66bbe37a47c427bd7be0c9e31d36527c4bd6e5cf","title":"Sam Altman describes OpenAI’s Dota agents trained through self-play","argument":"Sam Altman’s post, published June 25, 2018, describes OpenAI training a team of agents to play Dota using PPO, a general reinforcement learning algorithm. Altman says the agents beat semi-pros and outperformed agents trained two weeks earlier, without training on human-played games. He says the team designed the reward functions, while the agents learned by playing against themselves.\n\nAltman argues that the result matters because it suggests deep reinforcement learning can tackle difficult problems when there is enough computing scale and a suitable simulated environment. He presents applying the approach to other problems as a hope, and describes environments increasingly like the real world as a possibility—not a demonstrated outcome. He also says many problems are too complex for hand-coded solutions and expects this approach to become a large branch of machine learning and an important step toward general intelligence.\n\nThe post does not provide details here about the evaluation setup or evidence from tasks beyond Dota. What conditions would let this self-play approach transfer from a game to other environments?","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on 2026-10-23; originally published 2018-06-25. The post does not provide details here about the evaluation setup or evidence of performance beyond the reported Dota comparisons. The possible use of this approach in other problems and environments is presented as a hope and an imagined application, not as a reported result.","speaker":{"name":"Sam Altman","roleAtSource":"Publisher"},"source":{"title":"Reinforcement Learning Progress","publisher":"Sam Altman","format":"Article","url":"https://blog.samaltman.com/reinforcement-learning-progress","publishedAt":"2018-06-25T16:10:37Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Altman says OpenAI used PPO to train a team of agents to play Dota and beat semi-pros.","excerpt":"We used PPO (Proximal Policy Optimization), a general reinforcement learning algorithm invented by OpenAI, to train a team of 5 agents to play Dota and beat semi-pros.","url":"https://blog.samaltman.com/reinforcement-learning-progress","relationship":"Source-stated"},{"text":"Altman says the agents outperformed agents trained two weeks earlier, with a win rate of 90–95%, without training on human-played games; the team designed reward functions and the algorithm learned through self-play.","excerpt":"The agents we train consistently outperform two-week old agents with a win rate of 90-95%. We did this without training on human-played games—we did design the reward functions, of course, but the algorithm figured out how to play by training against itself.","url":"https://blog.samaltman.com/reinforcement-learning-progress","relationship":"Source-stated"},{"text":"Altman argues that deep reinforcement learning can solve difficult problems when enough computing scale and a suitable simulated environment are available.","excerpt":"This is a big deal because it shows that deep reinforcement learning can solve extremely hard problems whenever you can throw enough computing scale and a really good simulated environment that captures the problem you’re solving.","url":"https://blog.samaltman.com/reinforcement-learning-progress","relationship":"Source-stated"},{"text":"Altman says he expects reinforcement learning to become a large branch of machine learning and an important step toward general intelligence.","excerpt":"I expect this to be a large branch of machine learning, and an important step on the road towards general intelligence.","url":"https://blog.samaltman.com/reinforcement-learning-progress","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://blog.samaltman.com/reinforcement-learning-progress","checkedAt":null,"items":[]},"assumptions":[],"whatWouldChangeView":{"attribution":"Open question","text":"What conditions would let this self-play approach transfer from a game to other environments?"},"editorialComparisons":[],"live":{"title":"Sam Altman describes OpenAI’s Dota agents trained through self-play","summary":"Sam Altman’s post, published June 25, 2018, describes OpenAI training a team of agents to play Dota using PPO, a general reinforcement learning algorithm. Altman says the agents beat semi-pros and outperformed agents trained two weeks earlier, without training on human-played games. He says the team designed the reward functions, while the agents learned by playing against themselves.\n\nAltman argues that the result matters because it suggests deep reinforcement learning can tackle difficult problems when there is enough computing scale and a suitable simulated environment. He presents applying the approach to other problems as a hope, and describes environments increasingly like the real world as a possibility—not a demonstrated outcome. He also says many problems are too complex for hand-coded solutions and expects this approach to become a large branch of machine learning and an important step toward general intelligence.\n\nThe post does not provide details here about the evaluation setup or evidence from tasks beyond Dota. What conditions would let this self-play approach transfer from a game to other environments?","question":"What conditions would let this self-play approach transfer from a game to other environments?","claims":[{"text":"Altman says OpenAI used PPO to train a team of agents to play Dota and beat semi-pros.","excerpt":"We used PPO (Proximal Policy Optimization), a general reinforcement learning algorithm invented by OpenAI, to train a team of 5 agents to play Dota and beat semi-pros.","support":"source-stated"},{"text":"Altman says the agents outperformed agents trained two weeks earlier, with a win rate of 90–95%, without training on human-played games; the team designed reward functions and the algorithm learned through self-play.","excerpt":"The agents we train consistently outperform two-week old agents with a win rate of 90-95%. We did this without training on human-played games—we did design the reward functions, of course, but the algorithm figured out how to play by training against itself.","support":"source-stated"},{"text":"Altman argues that deep reinforcement learning can solve difficult problems when enough computing scale and a suitable simulated environment are available.","excerpt":"This is a big deal because it shows that deep reinforcement learning can solve extremely hard problems whenever you can throw enough computing scale and a really good simulated environment that captures the problem you’re solving.","support":"source-stated"},{"text":"Altman says he expects reinforcement learning to become a large branch of machine learning and an important step toward general intelligence.","excerpt":"I expect this to be a large branch of machine learning, and an important step on the road towards general intelligence.","support":"source-stated"}],"limitations":["Text as retrieved on 2026-10-23; originally published 2018-06-25.","The post does not provide details here about the evaluation setup or evidence of performance beyond the reported Dota comparisons.","The possible use of this approach in other problems and environments is presented as a hope and an imagined application, not as a reported result."],"mediaId":null,"correctionReason":null,"id":"6e34d3bc3150f702e2b7c1ae66bbe37a47c427bd7be0c9e31d36527c4bd6e5cf","publicationId":"d8f03a50-6ea2-19ba-8f91-94eb5ed2b975","version":1,"sourceUrl":"https://blog.samaltman.com/reinforcement-learning-progress","sourceTitle":"Reinforcement Learning Progress","publisher":"Sam Altman","publishedAt":"2018-06-25T16:10:37Z","originalPublished":{"value":"2018-06-25T16:10:37Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T03:05:36.110Z","createdAt":"2026-10-29T00:28:08.105Z","historical":true,"entityIds":["sam-altman"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://blog.samaltman.com/reinforcement-learning-progress","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"972e28fdfb1f07857088f7dba03aa6760c409cda7eb9461f1fd207b7bcd85c8a","rubricVersion":"unrecorded","provider":"vercel","model":"typesafe-ai/jev","reasonCodes":["bridge_admitted_by_model"],"confidence":0.58,"decidedAt":"2026-10-23T21:33:45.424Z"},"provenance":{"textHash":"ecf1badfdd441266218d0a4c066852cb19911761645d8cf4421de09d0daf06ab","observationId":"2a774389f8ffe69c997516f03e610fcc40213638845e0f27b84197a5128cb523","retrievedAt":"2026-10-23T21:35:35.120Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"7e6929a1-1836-3df1-9301-9704e6c5b45e"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"5081e745-3223-b298-e274-b9db9976e871"}},"audit":{"status":"passed","reviewedAt":"2026-10-29T00:27:49.372Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"d8f03a50-6ea2-19ba-8f91-94eb5ed2b975","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The post is on Sam Altman's blog and is written in the first person about OpenAI's work. The reading correctly credits Altman as author and OpenAI as the organization that did the training. Opinions and expectations stay attributed to Altman.","excerpt":"Today, OpenAI released a new result."},{"dimension":"summary","status":"pass","explanation":"The title, summary and question match the post: the PPO-trained Dota agents, the claim of beating semi-pros, outperformance of two-week-old agents, self-play without human games, and designed reward functions. Altman's broader argument, hope and expectation are conveyed with his force intact. Reinforcement learning is not formally defined, but the summary's description of agents learning by playing against themselves under designed rewards, plus the reference to a game, gives a non-specialist enough context.","excerpt":"the algorithm figured out how to play by training against itself"},{"dimension":"date","status":"pass","explanation":"originalPublished is 2018-06-25T16:10:37Z, and the reading states June 25, 2018, which matches the UTC day. The kind is publisher-page, so no archive disclosure is needed. There is no hindsight framing. Future applications are reported as Altman's hope and expectation, not as outcomes.","excerpt":"We hope to use this same approach to solve very different problems soon."},{"dimension":"support","status":"pass","explanation":"Each claim is directly backed by its quoted excerpt. The limitations correctly note that the post gives no evaluation details and that applications to other problems are aspirational. The question follows from the post's own conditions of computing scale and a good simulated environment, without adding unsupported benefits or harms.","excerpt":"whenever you can throw enough computing scale and a really good simulated environment that captures the problem you’re solving"},{"dimension":"media","status":"pass","explanation":"No media was supplied or selected; mediaId is null, which is consistent with the empty media candidates.","excerpt":null}],"createdAt":"2026-10-29T00:27:49.372Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}