Toby Ord · Oct 20, 2025
Toby Ord argues RL scaling is becoming costly to extend
Markdown · JSON · Original source
Toby Ord’s article was published on October 20, 2025. Ord distinguishes -scaling, which uses more Reinforcement Learning (RL) , from inference-scaling, which uses more compute while a model answers. He argues that the charts he examines show a much larger increase in RL compute is needed to match the performance gain from more : “a 10x scaling of RL is required to get the same performance boost as a 3x scaling of inference.” Ord says RL’s early gains were affordable because RL training started from a small base beside . As RL training compute approaches pre-training compute, he argues, that cost advantage fades. Inference-scaling has a different cost: it raises the ongoing expense of using a model, rather than the one-off cost of training it. Ord concludes that RL may have enabled longer , but its poor scaling leaves inference-scaling as the remaining form of compute-scaling. He warns that this may mean losing the ability to turn more compute into more intelligence. How well these scaling relationships hold for other companies’ models remains difficult to assess, he says, because training curves for cutting-edge models are often confidential.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
Toby Ord · Article
Source details
Original source date: Oct 20, 2025 (2025-10-20T18:23:37+01:00).
Date basis: Date stated on the publisher’s page.
Added to Escape Meridian: 2026-11-11T17:08:24.820Z. First observed: 2026-10-27T21:45:35.062Z. Retrieved: 2026-10-27T21:47:29.743Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
Some references could not be checked. Reload this page to try again.
Limits
Text as retrieved on October 27, 2026; originally published October 20, 2025. Ord notes that training curves for cutting-edge models from other companies are often confidential, limiting how well the pattern can be checked beyond . The article’s estimates and scaling relationships draw on charts and comparisons Ord discusses; the reading does not independently verify those underlying data or estimates.
Open question
What training curves from other companies would show whether -scaling follows the pattern Ord infers from ’s models?
Source-stated
Ord distinguishes RL-scaling, which increases RL training compute, from inference-scaling, which increases compute used while a model answers.
Scaling the amount of compute used for RL during training Scaling the amount of compute used for inference during deployment
Source-stated
Ord says that matching the performance boost from a threefold increase in inference compute takes a tenfold increase in RL compute.
a 10x scaling of RL is required to get the same performance boost as a 3x scaling of inference
Source-stated
Ord argues that early RL gains were affordable because RL training began from a small base compared with pre-training compute.
This is solely because the scaling of RL compute began from such a small base compared with the massive amount of pre-training compute invested in today’s models.
Source-stated
Ord argues that inference-scaling raises ongoing deployment costs, unlike training compute, which raises a one-off cost.
For one thing, it scales up the flow of ongoing costs instead of scaling the one-off training cost.
Source-stated
Ord says training curves for cutting-edge models from other companies are often confidential, making the pattern hard to verify beyond OpenAI.
It is hard to verify that this trend holds for models from other companies, as this data on training curves for cutting-edge models is often treated as confidential.
Source-stated
Ord concludes that RL may have enabled longer chain of thought, but its poor scaling may leave inference-scaling as the remaining form of compute-scaling.
This leaves us with inference-scaling as the remaining form of compute-scaling. RL helped enable inference-scaling via longer chain of thought
Source and contributor context
Toby Ord · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Reinforcement learning
Training by trial and error, in which a system is rewarded for the outcomes its designers want.
AI compute
The computing resources, meaning chips, data centers and electricity, used to train or run AI models.
Model inference
Using a trained model to produce a prediction or response to new input.
Pre-training
A model's initial training on a large dataset, before further adaptation for particular uses.
Chain of thought
The step-by-step working a model writes out before giving its final answer.
OpenAI
OpenAI develops AI models and products and publishes research, release announcements and evaluations.
Editorial provenance
Version 1. Publication ID: d1b634c5-bca3-e921-565d-6c6d973a753b.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-11T17:03:25.995Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.
Observation: 11dcf343ddb37a348ab4c5c585fdebefea88b5963a0a59c0b577260e10207412. Source text hash: 79b060a0465fcf9297f672a9dd3b95bac7a4eaf473f9ae49f5e717ab1f1f84b5.
Revision and source-check history
Version 1 · 2026-11-11T17:08:24.820Z
Toby Ord argues RL scaling is becoming costly to extend
Toby Ord’s article was published on October 20, 2025. Ord distinguishes -scaling, which uses more Reinforcement Learning (RL) , from inference-scaling, which uses more compute while a model answers. He argues that the charts he examines show a much larger increase in RL compute is needed to match the performance gain from more : “a 10x scaling of RL is required to get the same performance boost as a 3x scaling of inference.” Ord says RL’s early gains were affordable because RL training started from a small base beside . As RL training compute approaches pre-training compute, he argues, that cost advantage fades. Inference-scaling has a different cost: it raises the ongoing expense of using a model, rather than the one-off cost of training it. Ord concludes that RL may have enabled longer , but its poor scaling leaves inference-scaling as the remaining form of compute-scaling. He warns that this may mean losing the ability to turn more compute into more intelligence. How well these scaling relationships hold for other companies’ models remains difficult to assess, he says, because training curves for cutting-edge models are often confidential.
Original source
Toby Ord · Article
Source details
Original source date: Oct 20, 2025 (2025-10-20T18:23:37+01:00).
Date basis: Date stated on the publisher’s page.
Added to Escape Meridian: 2026-11-11T17:08:24.820Z. First observed: 2026-10-27T21:45:35.062Z. Retrieved: 2026-10-27T21:47:29.743Z.
Limits
Text as retrieved on October 27, 2026; originally published October 20, 2025. Ord notes that training curves for cutting-edge models from other companies are often confidential, limiting how well the pattern can be checked beyond . The article’s estimates and scaling relationships draw on charts and comparisons Ord discusses; the reading does not independently verify those underlying data or estimates.
Open question
What training curves from other companies would show whether -scaling follows the pattern Ord infers from ’s models?
Source-stated
Ord distinguishes RL-scaling, which increases RL training compute, from inference-scaling, which increases compute used while a model answers.
Scaling the amount of compute used for RL during training Scaling the amount of compute used for inference during deployment
Source-stated
Ord says that matching the performance boost from a threefold increase in inference compute takes a tenfold increase in RL compute.
a 10x scaling of RL is required to get the same performance boost as a 3x scaling of inference
Source-stated
Ord argues that early RL gains were affordable because RL training began from a small base compared with pre-training compute.
This is solely because the scaling of RL compute began from such a small base compared with the massive amount of pre-training compute invested in today’s models.
Source-stated
Ord argues that inference-scaling raises ongoing deployment costs, unlike training compute, which raises a one-off cost.
For one thing, it scales up the flow of ongoing costs instead of scaling the one-off training cost.
Source-stated
Ord says training curves for cutting-edge models from other companies are often confidential, making the pattern hard to verify beyond OpenAI.
It is hard to verify that this trend holds for models from other companies, as this data on training curves for cutting-edge models is often treated as confidential.
Source-stated
Ord concludes that RL may have enabled longer chain of thought, but its poor scaling may leave inference-scaling as the remaining form of compute-scaling.
This leaves us with inference-scaling as the remaining form of compute-scaling. RL helped enable inference-scaling via longer chain of thought
Source and contributor context
Toby Ord · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Reinforcement learning
Training by trial and error, in which a system is rewarded for the outcomes its designers want.
AI compute
The computing resources, meaning chips, data centers and electricity, used to train or run AI models.
Model inference
Using a trained model to produce a prediction or response to new input.
Pre-training
A model's initial training on a large dataset, before further adaptation for particular uses.
Chain of thought
The step-by-step working a model writes out before giving its final answer.
OpenAI
OpenAI develops AI models and products and publishes research, release announcements and evaluations.
Editorial provenance
Version 1. Publication ID: d1b634c5-bca3-e921-565d-6c6d973a753b.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-11T17:03:25.995Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.
Observation: 11dcf343ddb37a348ab4c5c585fdebefea88b5963a0a59c0b577260e10207412. Source text hash: 79b060a0465fcf9297f672a9dd3b95bac7a4eaf473f9ae49f5e717ab1f1f84b5.
passed · 2026-11-11T17:03:25.995Z · anthropic/claude-opus-5.5
Publication: d1b634c5-bca3-e921-565d-6c6d973a753b
attribution — pass: This is a single essay by Toby Ord on his own site, and the reading attributes its arguments to him throughout. Its forecasts and estimates are framed as his claims, not as established facts.
How Well Does RL Scale? October 20, 2025Toby Ord
summary — pass: The reading explains RL-scaling and inference-scaling in plain language, and the definitions cover RL, compute, inference, pre-training and chain of thought. It conveys Ord's central argument, the cost mechanism and his conclusion. Minor note: it softens some of his force. He calls the scaling breathtakingly inefficient and the end of the line for cheap RL-scaling. He also says RL helped enable longer chain of thought, which the reading hedges as may have enabled. These are tonal points, not a distortion of his position. The question follows from the source's own caveat.
This is breathtakingly inefficient scaling.
date — pass: The instant 2025-10-20T18:23:37+01:00 falls on October 20, 2025 in UTC, which matches the stated date and the source's dateline. The kind is publisher-page, so no archive disclosure is needed. Ord's estimates are presented as his arguments, not as later outcomes, and there is no hindsight framing.
I estimate that at the time of writing (Oct 2025)
support — pass: Each claim matches its quoted source passage. The statement that the cost advantage fades as RL compute approaches pre-training compute is supported by Ord's discussion of the point where RL reaches the size of pre-training. The closing warning about turning compute into intelligence is in the source. No benefits, costs or stakeholders are invented.
But this changes dramatically once RL-training reaches and then exceeds the size of the pre-training compute.
media — pass: No media was selected, and no verified media candidates were supplied, so nothing is misidentified.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
Training by trial and error, in which a system is rewarded for the outcomes its designers want.
Definition referenceThe computing resources, meaning chips, data centers and electricity, used to train or run AI models.
Definition referenceUsing a trained model to produce a prediction or response to new input.
Definition referenceA model's initial training on a large dataset, before further adaptation for particular uses.
Definition referenceThe step-by-step working a model writes out before giving its final answer.
Definition referenceOpenAI develops AI models and products and publishes research, release announcements and evaluations.
Explore OpenAITraining by trial and error, in which a system is rewarded for the outcomes its designers want.
Definition referenceOpenAI develops AI models and products and publishes research, release announcements and evaluations.
Explore OpenAITraining by trial and error, in which a system is rewarded for the outcomes its designers want.
Definition referenceThe computing resources, meaning chips, data centers and electricity, used to train or run AI models.
Definition referenceUsing a trained model to produce a prediction or response to new input.
Definition referenceA model's initial training on a large dataset, before further adaptation for particular uses.
Definition referenceThe step-by-step working a model writes out before giving its final answer.
Definition referenceOpenAI develops AI models and products and publishes research, release announcements and evaluations.
Explore OpenAITraining by trial and error, in which a system is rewarded for the outcomes its designers want.
Definition referenceOpenAI develops AI models and products and publishes research, release announcements and evaluations.
Explore OpenAI