# METR proposes a cost-based measure of AI optimization ability [Canonical reading](https://escapemeridian.com/readings/metr-proposes-a-cost-based-measure-of-ai-optimization-ability-cf14bd96) METR’s post, published July 21, 2026, proposes an “expenditure horizon” to compare how much improvement AI agents and people achieve for the same expenditure. The measure is where their cost-performance curves cross: the budget at which people become more cost-effective than agents. METR illustrates the approach with agents optimizing NanoGPT, a public project to reduce the training time of a language model. METR estimates that some tested models made modest, re-validated improvements, while GPT-5 and Opus-4.1 made no meaningful progress. The authors conclude that autonomous agent optimization had so far had minimal effect on AI R&D progress in NanoGPT. They distinguish that result from hybrid optimization, where people use AI assistance, which the study did not measure. The proposed measure depends on estimating human labor costs, which METR says are highly uncertain, and on results from a single optimization problem. The authors also warn that estimated horizons vary substantially with assumptions about the cost of human improvement. Whether the measure gives a useful comparison on other, more representative AI R&D problems remains unknown. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source METR · Article Original source date: Jul 21, 2026 (2026-07-21T07:00:00.000Z). Date basis: Date supplied by the publisher’s feed. Added to Escape Meridian: 2026-11-03T12:58:54.784Z. First observed: 2026-10-18T02:49:40.202Z. Retrieved: 2026-10-23T22:39:50.867Z. - [Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT](https://metr.org/blog/2026-07-21-expenditure-horizon/) ## Limits Text as retrieved on October 23, 2026; originally published July 21, 2026. The study is a proof of concept using NanoGPT; the evidence does not establish how the measure would perform across other AI R&D problems. METR says its human-labor estimates are highly uncertain, and the estimated expenditure horizons vary with assumptions about human improvement costs. The study compares autonomous agent and human optimization; it does not measure hybrid optimization. ## Open question How would expenditure horizons compare across other AI R&D problems, where human effort and the cost of verifying improvements may differ? ## Source-stated METR proposes expenditure horizon as a way to compare AI-agent and human optimization by cost. > We define an agent’s “expenditure horizon” over an optimization problem as the dollar value at which the improvement to the goal metric is equal to the improvement by a human with the same budget. - [Supporting source](https://metr.org/blog/2026-07-21-expenditure-horizon/) ## Source-stated METR estimates that the tested models’ expenditure horizons on NanoGPT ranged from $0 to $3,300. > We estimate expenditure horizons between \\$0 and \\$3,300 for models on the NanoGPT speedrun. - [Supporting source](https://metr.org/blog/2026-07-21-expenditure-horizon/) ## Source-stated METR reports that GPT-5 and Opus-4.1 made no meaningful progress after re-validation. > GPT-5 and Opus-4.1 did not make any meaningful progress at all. - [Supporting source](https://metr.org/blog/2026-07-21-expenditure-horizon/) ## Source-stated METR concludes that autonomous agent optimization had so far had minimal effect on AI R&D progress in NanoGPT, while noting that hybrid optimization was not ruled out. > Overall, although some models have expenditure horizons in the thousands of dollars, they are small relative to the overall human labor, indicating that autonomous agent optimization has so far had minimal effect on AI R&D progress in NanoGPT. - [Supporting source](https://metr.org/blog/2026-07-21-expenditure-horizon/) ## Source-stated METR says its human-effort estimates are highly uncertain. > These estimates are highly uncertain, especially due to estimating the value of human time — the true cost could be significantly higher or lower. - [Supporting source](https://metr.org/blog/2026-07-21-expenditure-horizon/) ## Source and contributor context METR · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## AI agent An AI system that chooses and carries out actions toward a goal, often using software tools. - [Definition reference](https://www.ibm.com/think/topics/ai-agents) ## METR METR studies what AI systems can do and how reliably they complete tasks, including work with human developers. - [Explore this context](/entities/metr) - [Reference](https://metr.org/) ## Editorial provenance Version 1. Publication ID: c8fed03f-331b-1b68-6fbd-39919bec40c6. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-03T12:58:53.133Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model. Observation: 5de45e9eb471298f35fb0d2a53634cd2754040ca72763ccdf6379895531e9af6. Source text hash: 72b85a23f5b2dea05f9e0f7ab275167377413d899e3124d5e244439f67334bc8. ## Revision and source-check history ### Version 1 — 2026-11-03T12:58:54.784Z METR’s post, published July 21, 2026, proposes an “expenditure horizon” to compare how much improvement AI agents and people achieve for the same expenditure. The measure is where their cost-performance curves cross: the budget at which people become more cost-effective than agents. METR illustrates the approach with agents optimizing NanoGPT, a public project to reduce the training time of a language model. METR estimates that some tested models made modest, re-validated improvements, while GPT-5 and Opus-4.1 made no meaningful progress. The authors conclude that autonomous agent optimization had so far had minimal effect on AI R&D progress in NanoGPT. They distinguish that result from hybrid optimization, where people use AI assistance, which the study did not measure. The proposed measure depends on estimating human labor costs, which METR says are highly uncertain, and on results from a single optimization problem. The authors also warn that estimated horizons vary substantially with assumptions about the cost of human improvement. Whether the measure gives a useful comparison on other, more representative AI R&D problems remains unknown. ## Original source METR · Article Original source date: Jul 21, 2026 (2026-07-21T07:00:00.000Z). Date basis: Date supplied by the publisher’s feed. Added to Escape Meridian: 2026-11-03T12:58:54.784Z. First observed: 2026-10-18T02:49:40.202Z. Retrieved: 2026-10-23T22:39:50.867Z. - [Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT](https://metr.org/blog/2026-07-21-expenditure-horizon/) ## Limits Text as retrieved on October 23, 2026; originally published July 21, 2026. The study is a proof of concept using NanoGPT; the evidence does not establish how the measure would perform across other AI R&D problems. METR says its human-labor estimates are highly uncertain, and the estimated expenditure horizons vary with assumptions about human improvement costs. The study compares autonomous agent and human optimization; it does not measure hybrid optimization. ## Open question How would expenditure horizons compare across other AI R&D problems, where human effort and the cost of verifying improvements may differ? ## Source-stated METR proposes expenditure horizon as a way to compare AI-agent and human optimization by cost. > We define an agent’s “expenditure horizon” over an optimization problem as the dollar value at which the improvement to the goal metric is equal to the improvement by a human with the same budget. - [Supporting source](https://metr.org/blog/2026-07-21-expenditure-horizon/) ## Source-stated METR estimates that the tested models’ expenditure horizons on NanoGPT ranged from $0 to $3,300. > We estimate expenditure horizons between \\$0 and \\$3,300 for models on the NanoGPT speedrun. - [Supporting source](https://metr.org/blog/2026-07-21-expenditure-horizon/) ## Source-stated METR reports that GPT-5 and Opus-4.1 made no meaningful progress after re-validation. > GPT-5 and Opus-4.1 did not make any meaningful progress at all. - [Supporting source](https://metr.org/blog/2026-07-21-expenditure-horizon/) ## Source-stated METR concludes that autonomous agent optimization had so far had minimal effect on AI R&D progress in NanoGPT, while noting that hybrid optimization was not ruled out. > Overall, although some models have expenditure horizons in the thousands of dollars, they are small relative to the overall human labor, indicating that autonomous agent optimization has so far had minimal effect on AI R&D progress in NanoGPT. - [Supporting source](https://metr.org/blog/2026-07-21-expenditure-horizon/) ## Source-stated METR says its human-effort estimates are highly uncertain. > These estimates are highly uncertain, especially due to estimating the value of human time — the true cost could be significantly higher or lower. - [Supporting source](https://metr.org/blog/2026-07-21-expenditure-horizon/) ## Source and contributor context METR · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## AI agent An AI system that chooses and carries out actions toward a goal, often using software tools. - [Definition reference](https://www.ibm.com/think/topics/ai-agents) ## METR METR studies what AI systems can do and how reliably they complete tasks, including work with human developers. - [Explore this context](/entities/metr) - [Reference](https://metr.org/) ## Editorial provenance Version 1. Publication ID: c8fed03f-331b-1b68-6fbd-39919bec40c6. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-03T12:58:53.133Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model. Observation: 5de45e9eb471298f35fb0d2a53634cd2754040ca72763ccdf6379895531e9af6. Source text hash: 72b85a23f5b2dea05f9e0f7ab275167377413d899e3124d5e244439f67334bc8. ### passed — 2026-11-03T12:58:53.133Z — anthropic/claude-opus-5.5 Publication: c8fed03f-331b-1b68-6fbd-39919bec40c6 attribution — pass: The reading attributes the proposal, estimates and conclusions to METR and its authors. This matches the METR blog post, a single argued piece rather than a listing. > We propose a measure of an AI agent’s optimization ability with an “expenditure horizon.” summary — pass: The reading names the actor and explains expenditure horizon as the point where cost-performance curves cross. It describes NanoGPT in plain terms and explains hybrid optimization. It states the main finding, the conclusion and the key uncertainties. The title and question follow from the source's own limitations. Central terms are understandable without specialist background. > the budget at which humans become more cost-effective than AIs date — pass: The displayed date, July 21, 2026, matches the UTC calendar day of evidence.originalPublished and the URL path. The evidence is a publisher page, not an archive copy. The reading uses no later events, presents no forecasts as outcomes, and has no hindsight framing. support — pass: Each claim's excerpt appears verbatim in the source and supports it. The source supports the single-problem caveat because it wishes for measurement across several problems. It supports the sensitivity caveat because it calls the horizon fairly sensitive to human-cost assumptions. The hybrid caveat holds because the method only estimates autonomous optimization. The closing unknown is framed as an open question, not a claim. > We show below that the estimated expenditure horizon is fairly sensitive to our assumption on returns to human effort. media — pass: The reading selects no media, and no verified media candidates were supplied, so there is no media identity or relevance issue. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/cf14bd96d649869d75c985abcc8aed63cc542dfa2d548a345039df797e350fa5/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/metr-proposes-a-cost-based-measure-of-ai-optimization-ability-cf14bd96.json)