# METR notes how modeling choices affect time horizon results [Canonical reading](https://escapemeridian.com/readings/metr-notes-how-modeling-choices-affect-time-horizon-results-755d32a1) METR’s note, published March 20, 2026, examines how choices in its time horizon model affect estimates of how long tasks AI models can complete at a given success rate. The author says many reasonable alternatives mostly lower recent models’ 50% time horizon estimates, while leaving them within very wide confidence intervals. The note highlights a modeling mistake involving regularization, which METR fixed, and finds that alternative success-rate curves can also shift estimates, especially for models near the task suite’s limits. The author’s main concern is uncertainty in task-length estimates: the analyses suggest noise could lower some estimates, but the author says its impact remains very uncertain. The note argues that the task distribution itself is a bigger source of uncertainty than analysis choices, and that human completion time is an imperfect proxy for difficulty for models. These caveats matter when interpreting the estimates: they describe results under particular tasks and modeling assumptions, not a general guarantee of what models can do. The author hopes to investigate task-length noise further. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source METR · Article Original source date: Mar 20, 2026 (2026-03-20T07:00:00.000Z). Date basis: Date supplied by the publisher’s feed. Added to Escape Meridian: 2026-11-07T21:03:26.453Z. First observed: 2026-10-18T02:59:23.028Z. Retrieved: 2026-10-23T21:29:41.580Z. - [Impact of modeling assumptions on time horizon results](https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/) ## Limits Text as retrieved on October 23, 2026; originally published March 20, 2026. METR describes these notes as unpolished research updates that have gone through less review than its research posts and do not necessarily reflect METR’s views as a whole. The author is not identified by name in the retrieved text. The author says the effect of noise in task-length estimates is highly uncertain and that the methods considered are not conclusive. ## Open question How much would accounting for noise in task-length estimates change the time horizon results, given the author’s uncertainty about both the noise and how to model its effects? ## Source-stated The author says reasonable modeling variations mostly reduce recent models’ 50% time horizon estimates, while leaving them within wide confidence intervals. > The ones explored in this post mostly reduced recent 50% time horizon estimates, but kept them within the (very wide) confidence intervals. - [Supporting source](https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/) ## Source-stated The author identifies noise in task-length estimates as a major uncertainty and says its impact on time horizon results remains very uncertain. > I don’t think either of the techniques considered in this section are conclusive, and overall still feel very uncertain about the impact noisy task-length estimates are having on time horizon results. - [Supporting source](https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/) ## Source-stated The author considers task distribution and the use of human completion time as a proxy for model difficulty bigger sources of uncertainty than analysis choices. > I still think the biggest sources of uncertainty are the task distribution itself and the fact that “time taken for humans” is a very imperfect proxy for difficulty-for-models. - [Supporting source](https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/) ## Source-stated The note says METR fixed a regularization mistake that had affected time horizon results. > One example of this was the recent update to fix a modeling mistake with regularization - [Supporting source](https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/) ## Source and contributor context METR · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Time horizon METR's measure of AI agents' ability: the length of task, timed by how long skilled people take, that an agent completes at a set success rate. - [Definition reference](https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/) ## Confidence interval A range of values consistent with the data at a stated level of confidence. - [Definition reference](https://en.wikipedia.org/wiki/Confidence_interval) ## METR METR studies what AI systems can do and how reliably they complete tasks, including work with human developers. - [Explore this context](/entities/metr) - [Reference](https://metr.org/) ## TIME TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services. - [Explore this context](/entities/time) - [Reference](https://time.com/) ## Editorial provenance Version 1. Publication ID: 284dc120-8215-098c-f8b2-ed89e6028ab0. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-07T20:58:58.150Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model. Observation: 20ba3a5c445c4fabb017a777ebdd4b91ef0a0e4ce523ea0b0b9da8f5b52ad271. Source text hash: 4f72572dbf7878861129b66feb9d717243582f0b43d2c8a6a9aa7f7ba23fae42. ## Revision and source-check history ### Version 1 — 2026-11-07T21:03:26.453Z METR’s note, published March 20, 2026, examines how choices in its time horizon model affect estimates of how long tasks AI models can complete at a given success rate. The author says many reasonable alternatives mostly lower recent models’ 50% time horizon estimates, while leaving them within very wide confidence intervals. The note highlights a modeling mistake involving regularization, which METR fixed, and finds that alternative success-rate curves can also shift estimates, especially for models near the task suite’s limits. The author’s main concern is uncertainty in task-length estimates: the analyses suggest noise could lower some estimates, but the author says its impact remains very uncertain. The note argues that the task distribution itself is a bigger source of uncertainty than analysis choices, and that human completion time is an imperfect proxy for difficulty for models. These caveats matter when interpreting the estimates: they describe results under particular tasks and modeling assumptions, not a general guarantee of what models can do. The author hopes to investigate task-length noise further. ## Original source METR · Article Original source date: Mar 20, 2026 (2026-03-20T07:00:00.000Z). Date basis: Date supplied by the publisher’s feed. Added to Escape Meridian: 2026-11-07T21:03:26.453Z. First observed: 2026-10-18T02:59:23.028Z. Retrieved: 2026-10-23T21:29:41.580Z. - [Impact of modeling assumptions on time horizon results](https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/) ## Limits Text as retrieved on October 23, 2026; originally published March 20, 2026. METR describes these notes as unpolished research updates that have gone through less review than its research posts and do not necessarily reflect METR’s views as a whole. The author is not identified by name in the retrieved text. The author says the effect of noise in task-length estimates is highly uncertain and that the methods considered are not conclusive. ## Open question How much would accounting for noise in task-length estimates change the time horizon results, given the author’s uncertainty about both the noise and how to model its effects? ## Source-stated The author says reasonable modeling variations mostly reduce recent models’ 50% time horizon estimates, while leaving them within wide confidence intervals. > The ones explored in this post mostly reduced recent 50% time horizon estimates, but kept them within the (very wide) confidence intervals. - [Supporting source](https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/) ## Source-stated The author identifies noise in task-length estimates as a major uncertainty and says its impact on time horizon results remains very uncertain. > I don’t think either of the techniques considered in this section are conclusive, and overall still feel very uncertain about the impact noisy task-length estimates are having on time horizon results. - [Supporting source](https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/) ## Source-stated The author considers task distribution and the use of human completion time as a proxy for model difficulty bigger sources of uncertainty than analysis choices. > I still think the biggest sources of uncertainty are the task distribution itself and the fact that “time taken for humans” is a very imperfect proxy for difficulty-for-models. - [Supporting source](https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/) ## Source-stated The note says METR fixed a regularization mistake that had affected time horizon results. > One example of this was the recent update to fix a modeling mistake with regularization - [Supporting source](https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/) ## Source and contributor context METR · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Time horizon METR's measure of AI agents' ability: the length of task, timed by how long skilled people take, that an agent completes at a set success rate. - [Definition reference](https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/) ## Confidence interval A range of values consistent with the data at a stated level of confidence. - [Definition reference](https://en.wikipedia.org/wiki/Confidence_interval) ## METR METR studies what AI systems can do and how reliably they complete tasks, including work with human developers. - [Explore this context](/entities/metr) - [Reference](https://metr.org/) ## TIME TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services. - [Explore this context](/entities/time) - [Reference](https://time.com/) ## Editorial provenance Version 1. Publication ID: 284dc120-8215-098c-f8b2-ed89e6028ab0. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-07T20:58:58.150Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model. Observation: 20ba3a5c445c4fabb017a777ebdd4b91ef0a0e4ce523ea0b0b9da8f5b52ad271. Source text hash: 4f72572dbf7878861129b66feb9d717243582f0b43d2c8a6a9aa7f7ba23fae42. ### passed — 2026-11-07T20:58:58.150Z — anthropic/claude-opus-5.5 Publication: 284dc120-8215-098c-f8b2-ed89e6028ab0 attribution — pass: The reading attributes the analysis to a METR note and its unnamed author. The limitations disclose that the author is not named and that METR notes are less reviewed and may not reflect METR's views as a whole. Claims stay attributed to the author. > They do not necessarily reflect the views of METR as a whole. summary — pass: The reading accurately conveys the central argument: reasonable modeling variations mostly lower recent 50% time horizon estimates but stay within wide confidence intervals. It also covers the regularization fix, sensitivity to curve choices, the author's uncertainty about task-length noise, and the view that task distribution matters more. Time horizon and confidence interval are defined on the page. Calling task-length noise the author's main concern is slightly loose, since the author calls it the aspect they are least certain about. The reading then correctly ranks task distribution as larger. The question follows from the evidence. > The aspect I feel least certain about is noise in the task length estimates, which I hope to look into more in the future. date — pass: March 20, 2026 matches originalPublished (2026-03-20T07:00Z) and the URL date. The source is a publisher page, so no archive disclosure is needed. There is no hindsight framing, and the plans to investigate further are reported as the author's hopes. > as the top-performing model by 50% time horizon as of 2026/03/20 support — pass: Each claim's excerpt is verbatim and supports it. The statement that alternative curves shift estimates most for models near the suite's edge is supported. So is the statement that noise could lower some estimates while its impact remains very uncertain. The closing interpretation that results reflect particular tasks and assumptions is a modest reading of the author's own caveats, not an invented consequence. > Opus 4.6’s time horizons when discussing the impact of various changes. This is due to the fact that as the top-performing model by 50% time horizon as of 2026/03/20 it is closest to the end of the task suite media — pass: No media was selected, and no verified media candidates were supplied, so leaving the media empty is appropriate. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/755d32a1f9b38a69eaba39579c045f68161aeb0acfc59c7185742745d5c3f154/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/metr-notes-how-modeling-choices-affect-time-horizon-results-755d32a1.json)