Sam Altman · Jun 25, 2018
Sam Altman describes OpenAI’s Dota agents trained through self-play
Markdown · JSON · Original source
Sam Altman’s post, published June 25, 2018, describes OpenAI training a team of agents to play Dota using PPO, a general reinforcement learning algorithm. Altman says the agents beat semi-pros and outperformed agents trained two weeks earlier, without training on human-played games. He says the team designed the reward functions, while the agents learned by playing against themselves. Altman argues that the result matters because it suggests deep reinforcement learning can tackle difficult problems when there is enough computing scale and a suitable simulated environment. He presents applying the approach to other problems as a hope, and describes environments increasingly like the real world as a possibility—not a demonstrated outcome. He also says many problems are too complex for hand-coded solutions and expects this approach to become a large branch of machine learning and an important step toward general intelligence. The post does not provide details here about the evaluation setup or evidence from tasks beyond Dota. What conditions would let this self-play approach transfer from a game to other environments?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
Sam Altman · Article
Source details
Original source date: Jun 25, 2018 (2018-06-25T16:10:37Z).
Date basis: Date supplied by the publisher’s feed.
Added to Escape Meridian: 2026-10-29T00:28:08.105Z. First observed: 2026-10-18T03:05:36.110Z. Retrieved: 2026-10-23T21:35:35.120Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
Text as retrieved on 2026-10-23; originally published 2018-06-25. The post does not provide details here about the evaluation setup or evidence of performance beyond the reported Dota comparisons. The possible use of this approach in other problems and environments is presented as a hope and an imagined application, not as a reported result.
Open question
What conditions would let this self-play approach transfer from a game to other environments?
Source-stated
Altman says OpenAI used PPO to train a team of agents to play Dota and beat semi-pros.
We used PPO (Proximal Policy Optimization), a general reinforcement learning algorithm invented by OpenAI, to train a team of 5 agents to play Dota and beat semi-pros.
Source-stated
Altman says the agents outperformed agents trained two weeks earlier, with a win rate of 90–95%, without training on human-played games; the team designed reward functions and the algorithm learned through self-play.
The agents we train consistently outperform two-week old agents with a win rate of 90-95%. We did this without training on human-played games—we did design the reward functions, of course, but the algorithm figured out how to play by training against itself.
Source-stated
Altman argues that deep reinforcement learning can solve difficult problems when enough computing scale and a suitable simulated environment are available.
This is a big deal because it shows that deep reinforcement learning can solve extremely hard problems whenever you can throw enough computing scale and a really good simulated environment that captures the problem you’re solving.
Source-stated
Altman says he expects reinforcement learning to become a large branch of machine learning and an important step toward general intelligence.
I expect this to be a large branch of machine learning, and an important step on the road towards general intelligence.
Source and contributor context
Sam Altman · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Reinforcement learning
Training by trial and error, in which a system is rewarded for the outcomes its designers want.
Machine learning
Methods that let computers learn patterns from examples instead of following rules written by hand.
Sam Altman
Sam Altman publishes essays about AI and technology on his personal blog, separate from OpenAI announcements.
OpenAI
OpenAI develops AI models and products and publishes research, release announcements and evaluations.
Editorial provenance
Version 1. Publication ID: d8f03a50-6ea2-19ba-8f91-94eb5ed2b975.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: passed by anthropic/claude-opus-5.5 on 2026-10-29T00:27:49.372Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model.
Observation: 2a774389f8ffe69c997516f03e610fcc40213638845e0f27b84197a5128cb523. Source text hash: ecf1badfdd441266218d0a4c066852cb19911761645d8cf4421de09d0daf06ab.
Revision and source-check history
Version 1 · 2026-10-29T00:28:08.105Z
Sam Altman describes OpenAI’s Dota agents trained through self-play
Sam Altman’s post, published June 25, 2018, describes OpenAI training a team of agents to play Dota using PPO, a general reinforcement learning algorithm. Altman says the agents beat semi-pros and outperformed agents trained two weeks earlier, without training on human-played games. He says the team designed the reward functions, while the agents learned by playing against themselves. Altman argues that the result matters because it suggests deep reinforcement learning can tackle difficult problems when there is enough computing scale and a suitable simulated environment. He presents applying the approach to other problems as a hope, and describes environments increasingly like the real world as a possibility—not a demonstrated outcome. He also says many problems are too complex for hand-coded solutions and expects this approach to become a large branch of machine learning and an important step toward general intelligence. The post does not provide details here about the evaluation setup or evidence from tasks beyond Dota. What conditions would let this self-play approach transfer from a game to other environments?
Original source
Sam Altman · Article
Source details
Original source date: Jun 25, 2018 (2018-06-25T16:10:37Z).
Date basis: Date supplied by the publisher’s feed.
Added to Escape Meridian: 2026-10-29T00:28:08.105Z. First observed: 2026-10-18T03:05:36.110Z. Retrieved: 2026-10-23T21:35:35.120Z.
Limits
Text as retrieved on 2026-10-23; originally published 2018-06-25. The post does not provide details here about the evaluation setup or evidence of performance beyond the reported Dota comparisons. The possible use of this approach in other problems and environments is presented as a hope and an imagined application, not as a reported result.
Open question
What conditions would let this self-play approach transfer from a game to other environments?
Source-stated
Altman says OpenAI used PPO to train a team of agents to play Dota and beat semi-pros.
We used PPO (Proximal Policy Optimization), a general reinforcement learning algorithm invented by OpenAI, to train a team of 5 agents to play Dota and beat semi-pros.
Source-stated
Altman says the agents outperformed agents trained two weeks earlier, with a win rate of 90–95%, without training on human-played games; the team designed reward functions and the algorithm learned through self-play.
The agents we train consistently outperform two-week old agents with a win rate of 90-95%. We did this without training on human-played games—we did design the reward functions, of course, but the algorithm figured out how to play by training against itself.
Source-stated
Altman argues that deep reinforcement learning can solve difficult problems when enough computing scale and a suitable simulated environment are available.
This is a big deal because it shows that deep reinforcement learning can solve extremely hard problems whenever you can throw enough computing scale and a really good simulated environment that captures the problem you’re solving.
Source-stated
Altman says he expects reinforcement learning to become a large branch of machine learning and an important step toward general intelligence.
I expect this to be a large branch of machine learning, and an important step on the road towards general intelligence.
Source and contributor context
Sam Altman · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Reinforcement learning
Training by trial and error, in which a system is rewarded for the outcomes its designers want.
Machine learning
Methods that let computers learn patterns from examples instead of following rules written by hand.
Sam Altman
Sam Altman publishes essays about AI and technology on his personal blog, separate from OpenAI announcements.
OpenAI
OpenAI develops AI models and products and publishes research, release announcements and evaluations.
Editorial provenance
Version 1. Publication ID: d8f03a50-6ea2-19ba-8f91-94eb5ed2b975.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: passed by anthropic/claude-opus-5.5 on 2026-10-29T00:27:49.372Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model.
Observation: 2a774389f8ffe69c997516f03e610fcc40213638845e0f27b84197a5128cb523. Source text hash: ecf1badfdd441266218d0a4c066852cb19911761645d8cf4421de09d0daf06ab.
passed · 2026-10-29T00:27:49.372Z · anthropic/claude-opus-5.5
Publication: d8f03a50-6ea2-19ba-8f91-94eb5ed2b975
attribution — pass: The post is on Sam Altman's blog and is written in the first person about OpenAI's work. The reading correctly credits Altman as author and OpenAI as the organization that did the training. Opinions and expectations stay attributed to Altman.
Today, OpenAI released a new result.
summary — pass: The title, summary and question match the post: the PPO-trained Dota agents, the claim of beating semi-pros, outperformance of two-week-old agents, self-play without human games, and designed reward functions. Altman's broader argument, hope and expectation are conveyed with his force intact. Reinforcement learning is not formally defined, but the summary's description of agents learning by playing against themselves under designed rewards, plus the reference to a game, gives a non-specialist enough context.
the algorithm figured out how to play by training against itself
date — pass: originalPublished is 2018-06-25T16:10:37Z, and the reading states June 25, 2018, which matches the UTC day. The kind is publisher-page, so no archive disclosure is needed. There is no hindsight framing. Future applications are reported as Altman's hope and expectation, not as outcomes.
We hope to use this same approach to solve very different problems soon.
support — pass: Each claim is directly backed by its quoted excerpt. The limitations correctly note that the post gives no evaluation details and that applications to other problems are aspirational. The question follows from the post's own conditions of computing scale and a good simulated environment, without adding unsupported benefits or harms.
whenever you can throw enough computing scale and a really good simulated environment that captures the problem you’re solving
media — pass: No media was supplied or selected; mediaId is null, which is consistent with the empty media candidates.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.