Escape Meridian

· Nov 21, 2026

Ramez Naam: The Most Important Charts in AI are Wrong

Markdown · JSON · Original source

argues that charts projecting rapid growth in AI task length overstate what AI can do in real work. task horizon measures performance by the length of tasks an AI completes at a stated success rate. Naam says and Google data on research work inside AI companies show much shorter task lengths, and that the gap from prominent forecasts has widened. He stresses that forecasters and benchmark creators lacked this real-world evidence when they built their models. Naam does not argue that AI capabilities are static. His analysis finds exponential improvement in real-world task length plausible, but not conclusive at 80% success. He argues that high-reliability performance matters for autonomous work: success on shorter or less reliable tasks does not establish the ability to chain long tasks without human involvement. His higher-reliability estimates are speculative, and he says the available data do not show a self-sustaining AI self-improvement loop. Naam calls for AI companies to release detailed, longitudinal data so outside researchers can assess real-world use and update forecasts. His analysis draws mainly on OpenAI data, with more limited data from Google; he says has not released comparable detail.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

At 80% success, model-matched ECI → METR estimates are 33 times OpenAI’s internal horizon and 41 times Google’s. The original AI 2027 scenario gives ratios of…
Ramez Naam · At 80% success, model-matched ECI → METR estimates are 33 times OpenAI’s internal horizon and 41 times Google’s. The original AI 2027 scenario gives ratios of… · Open original source

Original source

· Article

Source details

Original source date: Nov 21, 2026 (2026-11-21T00:04:52+00:00).

Date basis: Date stated by the publisher.

Added to Escape Meridian: 2026-11-21T00:31:54.268Z. First observed: 2026-11-21T00:19:06.050Z. Retrieved: 2026-11-21T00:30:56.725Z.

Referenced by

Recorded citations in public readings. A reference alone does not establish agreement or influence.

No public references recorded here yet.

Reference data (JSON)

Limits

The real-world data discussed are primarily from , with a similar but limited Google comparison; Naam says has not released data at comparable detail or clarity. The higher-reliability task lengths are estimates from fitted curves, not direct measurements; Naam says there is insufficient data to measure task length at 95% or 99% success. The OpenAI task-length is adjusted for possible ambiguous or unsolvable tasks and tool errors; Naam notes that this adjustment may not capture all possible error.

Open question

What do longer, more detailed real-world data show about AI task lengths at higher success rates?

Source-stated

Naam argues that influential charts based on METR task horizon and AI forecasts overstate AI capabilities on real-world work.

Real world data inside of OpenAI and Google demonstrates that these charts are wrong.

Source-stated

Naam reports that internal AI research tasks reached about nine minutes at Google and an adjusted 16 minutes at OpenAI at 80% success.

AI models are 80% successful in accomplishing tasks of around nine minutes at Google and an adjusted 16 minutes at OpenAI.

Source-stated

Naam says exponential improvement in real-world task length is a working assumption, not a settled result, especially at 80% success.

Exponential growth is now our working assumption, but not yet conclusive.

Source-stated

Naam argues that the reported data do not show a self-sustaining AI self-improvement loop, while noting that the analysis does not account for possible changes in experiment quality.

That estimated gain is still well short of the 15-19% threshold for a self-sustaining AI recursive self-improvement loop, but it’s closer.

Source-stated

Naam calls for AI companies to publish detailed data that enable outside assessment of AI progress and real-world use.

The bedrock of any AI safety policy, in my mind, should be transparency.

Source and contributor context

Ramez Naam · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

Media declared by the original publisher; availability checked separately from the editorial reading.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

Ramez Naam

Explicitly tests self-improvement and diminishing-return assumptions; connects AI to energy and economics.

METR

METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.

OpenAI

OpenAI develops AI models and products and publishes research, release announcements and evaluations.

Anthropic

Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems.

Figure

Figure develops humanoid robots and publishes its own tests of how they perform physical tasks.

Editorial provenance

Version 1. Publication ID: d63efd43-eab1-4d31-8bf2-fcb5c2aac54b.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-21T00:31:54.268Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: useful_perspective.

Observation: cf60809b3e8be9219ef5941e2623531fab61d0db17c41274dfcdfd63abeac799. Source text hash: b53466d257c3556f63df3d8134837a15d61c45bb2a38c8fac76139e9366ba54a.

Revision and source-check history

Version 1 · 2026-11-21T00:31:54.268Z

Ramez Naam: The Most Important Charts in AI are Wrong

argues that charts projecting rapid growth in AI task length overstate what AI can do in real work. task horizon measures performance by the length of tasks an AI completes at a stated success rate. Naam says and Google data on research work inside AI companies show much shorter task lengths, and that the gap from prominent forecasts has widened. He stresses that forecasters and benchmark creators lacked this real-world evidence when they built their models. Naam does not argue that AI capabilities are static. His analysis finds exponential improvement in real-world task length plausible, but not conclusive at 80% success. He argues that high-reliability performance matters for autonomous work: success on shorter or less reliable tasks does not establish the ability to chain long tasks without human involvement. His higher-reliability estimates are speculative, and he says the available data do not show a self-sustaining AI self-improvement loop. Naam calls for AI companies to release detailed, longitudinal data so outside researchers can assess real-world use and update forecasts. His analysis draws mainly on OpenAI data, with more limited data from Google; he says has not released comparable detail.

Original source

· Article

Source details

Original source date: Nov 21, 2026 (2026-11-21T00:04:52+00:00).

Date basis: Date stated by the publisher.

Added to Escape Meridian: 2026-11-21T00:31:54.268Z. First observed: 2026-11-21T00:19:06.050Z. Retrieved: 2026-11-21T00:30:56.725Z.

Limits

The real-world data discussed are primarily from , with a similar but limited Google comparison; Naam says has not released data at comparable detail or clarity. The higher-reliability task lengths are estimates from fitted curves, not direct measurements; Naam says there is insufficient data to measure task length at 95% or 99% success. The OpenAI task-length is adjusted for possible ambiguous or unsolvable tasks and tool errors; Naam notes that this adjustment may not capture all possible error.

Open question

What do longer, more detailed real-world data show about AI task lengths at higher success rates?

Source-stated

Naam argues that influential charts based on METR task horizon and AI forecasts overstate AI capabilities on real-world work.

Real world data inside of OpenAI and Google demonstrates that these charts are wrong.

Source-stated

Naam reports that internal AI research tasks reached about nine minutes at Google and an adjusted 16 minutes at OpenAI at 80% success.

AI models are 80% successful in accomplishing tasks of around nine minutes at Google and an adjusted 16 minutes at OpenAI.

Source-stated

Naam says exponential improvement in real-world task length is a working assumption, not a settled result, especially at 80% success.

Exponential growth is now our working assumption, but not yet conclusive.

Source-stated

Naam argues that the reported data do not show a self-sustaining AI self-improvement loop, while noting that the analysis does not account for possible changes in experiment quality.

That estimated gain is still well short of the 15-19% threshold for a self-sustaining AI recursive self-improvement loop, but it’s closer.

Source-stated

Naam calls for AI companies to publish detailed data that enable outside assessment of AI progress and real-world use.

The bedrock of any AI safety policy, in my mind, should be transparency.

Source and contributor context

Ramez Naam · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

Media declared by the original publisher; availability checked separately from the editorial reading.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

Ramez Naam

Explicitly tests self-improvement and diminishing-return assumptions; connects AI to energy and economics.

METR

METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.

OpenAI

OpenAI develops AI models and products and publishes research, release announcements and evaluations.

Anthropic

Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems.

Figure

Figure develops humanoid robots and publishes its own tests of how they perform physical tasks.

Editorial provenance

Version 1. Publication ID: d63efd43-eab1-4d31-8bf2-fcb5c2aac54b.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-21T00:31:54.268Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: useful_perspective.

Observation: cf60809b3e8be9219ef5941e2623531fab61d0db17c41274dfcdfd63abeac799. Source text hash: b53466d257c3556f63df3d8134837a15d61c45bb2a38c8fac76139e9366ba54a.

passed · 2026-11-21T00:31:54.268Z · anthropic/claude-opus-5.5

Publication: d63efd43-eab1-4d31-8bf2-fcb5c2aac54b

attribution — pass: The reading attributes the argument, adjusted figures, projections and transparency call to Ramez Naam, the author and publisher of the post. Figures from OpenAI, Google and METR are framed as Naam's reading of their data, not as established facts.

Real world data inside of OpenAI and Google demonstrates that these charts are wrong.

summary — pass: The summary leads with Naam's thesis and explains METR task horizon in plain language. It notes that the data cover research work inside AI companies and preserves his caveats on exponential growth, reliability and self-improvement. Two passages are softer than the source but not wrong. The reading omits the gap sizes (33x to 41x, and over 100x for AI 2027). It also reduces his sharp criticism of Anthropic for forcing METR to redact evidence to a note that Anthropic released less detail. Restoring one gap figure and the redaction point would recover his force. The question is open and follows from the evidence.

At 80% success, ECI → METR estimates are 33× OpenAI’s internal task length and 41× Google’s.

date — pass: The original publication value, 2026-11-21T00:04:52Z, matches the post's own byline date of Nov 21, 2026 and the feed entry. There is no sign of an earlier first publication.

Ramez Naam Nov 21, 2026

support — pass: Each claim's excerpt is verbatim and supports the claim, kept as Naam's assertion. Caveats are source-grounded: the analysis ignores experiment quality, the high-reliability figures come from fitted curves, the OpenAI adjustment may be incomplete, and the data mostly come from OpenAI. Calling exponential growth plausible slightly understates his view that it now looks most likely, but that is not a contradiction. No benefits, costs or forecasts are invented.

We don’t have sufficient data to actually measure AI task length at 95% or 99% success rate but plausible extrapolations suggest it will be quite low.

media — pass: The selected media is the verified direct link to the original Substack post, matching the source URL. It is the most direct way for a reader to reach the source.

Challenge or correct this reading

Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.