# Toby Ord argues RL scaling is becoming costly to extend [Canonical reading](https://escapemeridian.com/readings/toby-ord-argues-rl-scaling-is-becoming-costly-to-extend-2954ee92) Toby Ord’s article was published on October 20, 2025. Ord distinguishes RL-scaling, which uses more Reinforcement Learning (RL) training compute, from inference-scaling, which uses more compute while a model answers. He argues that the charts he examines show a much larger increase in RL compute is needed to match the performance gain from more inference compute: “a 10x scaling of RL is required to get the same performance boost as a 3x scaling of inference.” Ord says RL’s early gains were affordable because RL training started from a small base beside pre-training. As RL training compute approaches pre-training compute, he argues, that cost advantage fades. Inference-scaling has a different cost: it raises the ongoing expense of using a model, rather than the one-off cost of training it. Ord concludes that RL may have enabled longer chain of thought, but its poor scaling leaves inference-scaling as the remaining form of compute-scaling. He warns that this may mean losing the ability to turn more compute into more intelligence. How well these scaling relationships hold for other companies’ models remains difficult to assess, he says, because training curves for cutting-edge models are often confidential. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source Toby Ord · Article Original source date: Oct 20, 2025 (2025-10-20T18:23:37+01:00). Date basis: Date stated on the publisher’s page. Added to Escape Meridian: 2026-11-11T17:08:24.820Z. First observed: 2026-10-27T21:45:35.062Z. Retrieved: 2026-10-27T21:47:29.743Z. - [How Well Does RL Scale?](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Limits Text as retrieved on October 27, 2026; originally published October 20, 2025. Ord notes that training curves for cutting-edge models from other companies are often confidential, limiting how well the pattern can be checked beyond OpenAI. The article’s estimates and scaling relationships draw on charts and comparisons Ord discusses; the reading does not independently verify those underlying data or estimates. ## Open question What training curves from other companies would show whether RL-scaling follows the pattern Ord infers from OpenAI’s models? ## Source-stated Ord distinguishes RL-scaling, which increases RL training compute, from inference-scaling, which increases compute used while a model answers. > Scaling the amount of compute used for RL during training > Scaling the amount of compute used for inference during deployment - [Supporting source](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Source-stated Ord says that matching the performance boost from a threefold increase in inference compute takes a tenfold increase in RL compute. > a 10x scaling of RL is required to get the same performance boost as a 3x scaling of inference - [Supporting source](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Source-stated Ord argues that early RL gains were affordable because RL training began from a small base compared with pre-training compute. > This is solely because the scaling of RL compute began from such a small base compared with the massive amount of pre-training compute invested in today’s models. - [Supporting source](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Source-stated Ord argues that inference-scaling raises ongoing deployment costs, unlike training compute, which raises a one-off cost. > For one thing, it scales up the flow of ongoing costs instead of scaling the one-off training cost. - [Supporting source](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Source-stated Ord says training curves for cutting-edge models from other companies are often confidential, making the pattern hard to verify beyond OpenAI. > It is hard to verify that this trend holds for models from other companies, as this data on training curves for cutting-edge models is often treated as confidential. - [Supporting source](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Source-stated Ord concludes that RL may have enabled longer chain of thought, but its poor scaling may leave inference-scaling as the remaining form of compute-scaling. > This leaves us with inference-scaling as the remaining form of compute-scaling. RL helped enable inference-scaling via longer chain of thought - [Supporting source](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Source and contributor context Toby Ord · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Reinforcement learning Training by trial and error, in which a system is rewarded for the outcomes its designers want. - [Definition reference](https://developers.google.com/machine-learning/glossary#reinforcement-learning-rl) ## AI compute The computing resources, meaning chips, data centers and electricity, used to train or run AI models. - [Definition reference](https://developers.google.com/machine-learning/glossary#compute) ## Model inference Using a trained model to produce a prediction or response to new input. - [Definition reference](https://developers.google.com/machine-learning/glossary#inference) ## Pre-training A model's initial training on a large dataset, before further adaptation for particular uses. - [Definition reference](https://developers.google.com/machine-learning/glossary#pre-training) ## Chain of thought The step-by-step working a model writes out before giving its final answer. - [Definition reference](https://developers.google.com/machine-learning/glossary#chain-of-thought-prompting) ## OpenAI OpenAI develops AI models and products and publishes research, release announcements and evaluations. - [Explore this context](/entities/openai) - [Reference](https://openai.com/news/) ## Editorial provenance Version 1. Publication ID: d1b634c5-bca3-e921-565d-6c6d973a753b. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-11T17:03:25.995Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model, second\_opinion\_categorical. Observation: 11dcf343ddb37a348ab4c5c585fdebefea88b5963a0a59c0b577260e10207412. Source text hash: 79b060a0465fcf9297f672a9dd3b95bac7a4eaf473f9ae49f5e717ab1f1f84b5. ## Revision and source-check history ### Version 1 — 2026-11-11T17:08:24.820Z Toby Ord’s article was published on October 20, 2025. Ord distinguishes RL-scaling, which uses more Reinforcement Learning (RL) training compute, from inference-scaling, which uses more compute while a model answers. He argues that the charts he examines show a much larger increase in RL compute is needed to match the performance gain from more inference compute: “a 10x scaling of RL is required to get the same performance boost as a 3x scaling of inference.” Ord says RL’s early gains were affordable because RL training started from a small base beside pre-training. As RL training compute approaches pre-training compute, he argues, that cost advantage fades. Inference-scaling has a different cost: it raises the ongoing expense of using a model, rather than the one-off cost of training it. Ord concludes that RL may have enabled longer chain of thought, but its poor scaling leaves inference-scaling as the remaining form of compute-scaling. He warns that this may mean losing the ability to turn more compute into more intelligence. How well these scaling relationships hold for other companies’ models remains difficult to assess, he says, because training curves for cutting-edge models are often confidential. ## Original source Toby Ord · Article Original source date: Oct 20, 2025 (2025-10-20T18:23:37+01:00). Date basis: Date stated on the publisher’s page. Added to Escape Meridian: 2026-11-11T17:08:24.820Z. First observed: 2026-10-27T21:45:35.062Z. Retrieved: 2026-10-27T21:47:29.743Z. - [How Well Does RL Scale?](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Limits Text as retrieved on October 27, 2026; originally published October 20, 2025. Ord notes that training curves for cutting-edge models from other companies are often confidential, limiting how well the pattern can be checked beyond OpenAI. The article’s estimates and scaling relationships draw on charts and comparisons Ord discusses; the reading does not independently verify those underlying data or estimates. ## Open question What training curves from other companies would show whether RL-scaling follows the pattern Ord infers from OpenAI’s models? ## Source-stated Ord distinguishes RL-scaling, which increases RL training compute, from inference-scaling, which increases compute used while a model answers. > Scaling the amount of compute used for RL during training > Scaling the amount of compute used for inference during deployment - [Supporting source](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Source-stated Ord says that matching the performance boost from a threefold increase in inference compute takes a tenfold increase in RL compute. > a 10x scaling of RL is required to get the same performance boost as a 3x scaling of inference - [Supporting source](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Source-stated Ord argues that early RL gains were affordable because RL training began from a small base compared with pre-training compute. > This is solely because the scaling of RL compute began from such a small base compared with the massive amount of pre-training compute invested in today’s models. - [Supporting source](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Source-stated Ord argues that inference-scaling raises ongoing deployment costs, unlike training compute, which raises a one-off cost. > For one thing, it scales up the flow of ongoing costs instead of scaling the one-off training cost. - [Supporting source](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Source-stated Ord says training curves for cutting-edge models from other companies are often confidential, making the pattern hard to verify beyond OpenAI. > It is hard to verify that this trend holds for models from other companies, as this data on training curves for cutting-edge models is often treated as confidential. - [Supporting source](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Source-stated Ord concludes that RL may have enabled longer chain of thought, but its poor scaling may leave inference-scaling as the remaining form of compute-scaling. > This leaves us with inference-scaling as the remaining form of compute-scaling. RL helped enable inference-scaling via longer chain of thought - [Supporting source](https://www.tobyord.com/writing/how-well-does-rl-scale) ## Source and contributor context Toby Ord · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Reinforcement learning Training by trial and error, in which a system is rewarded for the outcomes its designers want. - [Definition reference](https://developers.google.com/machine-learning/glossary#reinforcement-learning-rl) ## AI compute The computing resources, meaning chips, data centers and electricity, used to train or run AI models. - [Definition reference](https://developers.google.com/machine-learning/glossary#compute) ## Model inference Using a trained model to produce a prediction or response to new input. - [Definition reference](https://developers.google.com/machine-learning/glossary#inference) ## Pre-training A model's initial training on a large dataset, before further adaptation for particular uses. - [Definition reference](https://developers.google.com/machine-learning/glossary#pre-training) ## Chain of thought The step-by-step working a model writes out before giving its final answer. - [Definition reference](https://developers.google.com/machine-learning/glossary#chain-of-thought-prompting) ## OpenAI OpenAI develops AI models and products and publishes research, release announcements and evaluations. - [Explore this context](/entities/openai) - [Reference](https://openai.com/news/) ## Editorial provenance Version 1. Publication ID: d1b634c5-bca3-e921-565d-6c6d973a753b. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-11T17:03:25.995Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model, second\_opinion\_categorical. Observation: 11dcf343ddb37a348ab4c5c585fdebefea88b5963a0a59c0b577260e10207412. Source text hash: 79b060a0465fcf9297f672a9dd3b95bac7a4eaf473f9ae49f5e717ab1f1f84b5. ### passed — 2026-11-11T17:03:25.995Z — anthropic/claude-opus-5.5 Publication: d1b634c5-bca3-e921-565d-6c6d973a753b attribution — pass: This is a single essay by Toby Ord on his own site, and the reading attributes its arguments to him throughout. Its forecasts and estimates are framed as his claims, not as established facts. > How Well Does RL Scale? October 20, 2025Toby Ord summary — pass: The reading explains RL-scaling and inference-scaling in plain language, and the definitions cover RL, compute, inference, pre-training and chain of thought. It conveys Ord's central argument, the cost mechanism and his conclusion. Minor note: it softens some of his force. He calls the scaling breathtakingly inefficient and the end of the line for cheap RL-scaling. He also says RL helped enable longer chain of thought, which the reading hedges as may have enabled. These are tonal points, not a distortion of his position. The question follows from the source's own caveat. > This is breathtakingly inefficient scaling. date — pass: The instant 2025-10-20T18:23:37+01:00 falls on October 20, 2025 in UTC, which matches the stated date and the source's dateline. The kind is publisher-page, so no archive disclosure is needed. Ord's estimates are presented as his arguments, not as later outcomes, and there is no hindsight framing. > I estimate that at the time of writing (Oct 2025) support — pass: Each claim matches its quoted source passage. The statement that the cost advantage fades as RL compute approaches pre-training compute is supported by Ord's discussion of the point where RL reaches the size of pre-training. The closing warning about turning compute into intelligence is in the source. No benefits, costs or stakeholders are invented. > But this changes dramatically once RL-training reaches and then exceeds the size of the pre-training compute. media — pass: No media was selected, and no verified media candidates were supplied, so nothing is misidentified. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/2954ee927aa2e0b69b7f1851a9431b5240c5ec8766f594514ab655c33369f850/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/toby-ord-argues-rl-scaling-is-becoming-costly-to-extend-2954ee92.json)