· Mar 15, 2024
METR measures how post-training and agent improvements changed task performance
Markdown · JSON · Original source
’s report was published on March 15, 2024. It examines how post-training and further efforts to improve agents affected performance on a task set, and argues that evaluations should allow a safety margin for dangerous capabilities that further enhancements could unlock. On its tasks, GPT-4 post-training raised performance from 5±2% at the base model to 30±6% at the 0613 release. METR’s additional efforts raised it to 38±6%, an increase the report says was not . METR describes the post-training gain as comparable to moving from GPT-3.5 Turbo to GPT-4 on the tested agents. METR says this is some evidence that dramatically increasing capability may be difficult after competent for agency, but its results are inconclusive and it expects substantial headroom to remain. The task suite was not designed to test dangerous autonomous capabilities, so the experiments do not directly establish how much further elicitation could change results on such tests. The report lists reasons its estimates could be too high or too low.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
· Article
Source details
Original source date: Mar 15, 2024 (2024-03-15T08:00:00.000Z).
Date basis: Date supplied by the publisher’s feed.
Added to Escape Meridian: 2026-11-21T00:23:10.182Z. First observed: 2026-10-18T03:55:21.264Z. Retrieved: 2026-10-24T03:55:35.948Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
Text as retrieved on October 24, 2026; originally published March 15, 2024. The task suite was not designed specifically to test dangerous autonomous capabilities, so the report treats its results as evidence about such capabilities rather than a direct test. describes the experiments as preliminary and says its estimates are not conclusive; it lists possible reasons they could be too high or too low.
Open question
How would the measured effect of additional capability elicitation differ on focused on dangerous capabilities?
Source-stated
METR argues that evaluation protocols should include a safety margin for dangerous capabilities that further post-training or external efforts could unlock.
Our example evaluation protocol suggests adding safety margin to take into account increases in dangerous capabilities that could be unlocked by further post-training enhancements.
Source-stated
In METR’s task set, GPT-4 post-training raised performance from 5±2% at the base model to 30±6% at the 0613 release.
OpenAI’s post-training had a large effect, going from 5±2% at the GPT-4 base model to 30±6% at the 0613 release.
Source-stated
METR’s additional agent-improvement efforts raised performance from 30±6% to 38±6%, an increase the report says was not statistically significant.
We see a smaller gap from our agent improvements, going from 30±6% to 38±6% at the most sophisticated all tools + RM version (not statistically significant)
Source-stated
METR says the task suite was not specifically designed to test dangerous autonomous capabilities.
This task suite is not intended to be a test of dangerous autonomous capabilities in particular.
Source-stated
METR says its results are inconclusive and that substantial headroom may remain.
but our results are not conclusive. We expect substantial headroom to remain.
Source and contributor context
METR · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Statistical significance
A test of whether a result is unlikely to be due to chance alone; it does not say how large or important an effect is.
Fine-tuning
Further training of an existing model to adapt it to a particular task or use.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
METR
METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Editorial provenance
Version 1. Publication ID: f30150f9-d8f7-f06a-6a48-7e3a976fa480.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-21T00:18:42.894Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model.
Observation: 2f34699989e510aa7691a40455ed2b9bba9651fce898cfea0b8d5a3103113265. Source text hash: 009109922e4e3bb802b19a5cdf1a6b1f09a597c3b2c35525b7c46bb0b7513cf8.
Revision and source-check history
Version 1 · 2026-11-21T00:23:10.182Z
METR measures how post-training and agent improvements changed task performance
’s report was published on March 15, 2024. It examines how post-training and further efforts to improve agents affected performance on a task set, and argues that evaluations should allow a safety margin for dangerous capabilities that further enhancements could unlock. On its tasks, GPT-4 post-training raised performance from 5±2% at the base model to 30±6% at the 0613 release. METR’s additional efforts raised it to 38±6%, an increase the report says was not . METR describes the post-training gain as comparable to moving from GPT-3.5 Turbo to GPT-4 on the tested agents. METR says this is some evidence that dramatically increasing capability may be difficult after competent for agency, but its results are inconclusive and it expects substantial headroom to remain. The task suite was not designed to test dangerous autonomous capabilities, so the experiments do not directly establish how much further elicitation could change results on such tests. The report lists reasons its estimates could be too high or too low.
Original source
· Article
Source details
Original source date: Mar 15, 2024 (2024-03-15T08:00:00.000Z).
Date basis: Date supplied by the publisher’s feed.
Added to Escape Meridian: 2026-11-21T00:23:10.182Z. First observed: 2026-10-18T03:55:21.264Z. Retrieved: 2026-10-24T03:55:35.948Z.
Limits
Text as retrieved on October 24, 2026; originally published March 15, 2024. The task suite was not designed specifically to test dangerous autonomous capabilities, so the report treats its results as evidence about such capabilities rather than a direct test. describes the experiments as preliminary and says its estimates are not conclusive; it lists possible reasons they could be too high or too low.
Open question
How would the measured effect of additional capability elicitation differ on focused on dangerous capabilities?
Source-stated
METR argues that evaluation protocols should include a safety margin for dangerous capabilities that further post-training or external efforts could unlock.
Our example evaluation protocol suggests adding safety margin to take into account increases in dangerous capabilities that could be unlocked by further post-training enhancements.
Source-stated
In METR’s task set, GPT-4 post-training raised performance from 5±2% at the base model to 30±6% at the 0613 release.
OpenAI’s post-training had a large effect, going from 5±2% at the GPT-4 base model to 30±6% at the 0613 release.
Source-stated
METR’s additional agent-improvement efforts raised performance from 30±6% to 38±6%, an increase the report says was not statistically significant.
We see a smaller gap from our agent improvements, going from 30±6% to 38±6% at the most sophisticated all tools + RM version (not statistically significant)
Source-stated
METR says the task suite was not specifically designed to test dangerous autonomous capabilities.
This task suite is not intended to be a test of dangerous autonomous capabilities in particular.
Source-stated
METR says its results are inconclusive and that substantial headroom may remain.
but our results are not conclusive. We expect substantial headroom to remain.
Source and contributor context
METR · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Statistical significance
A test of whether a result is unlikely to be due to chance alone; it does not say how large or important an effect is.
Fine-tuning
Further training of an existing model to adapt it to a particular task or use.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
METR
METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Editorial provenance
Version 1. Publication ID: f30150f9-d8f7-f06a-6a48-7e3a976fa480.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-21T00:18:42.894Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model.
Observation: 2f34699989e510aa7691a40455ed2b9bba9651fce898cfea0b8d5a3103113265. Source text hash: 009109922e4e3bb802b19a5cdf1a6b1f09a597c3b2c35525b7c46bb0b7513cf8.
uncertain · 2026-11-21T00:18:42.894Z · anthropic/claude-opus-5.5
Publication: f30150f9-d8f7-f06a-6a48-7e3a976fa480
attribution — pass: The reading consistently attributes its findings, arguments and caveats to METR's single report. It does not present them as established fact. OpenAI's post-training is correctly identified as the source of the GPT-4 gain.
We use a series of agents built on versions of OpenAI GPT-3.5 Turbo and GPT-4 for our case study.
summary — uncertain: The numbers, the caveats and METR's tentative conclusion are reported faithfully. However, the central terms are never explained in plain language. Post-training, agent (a model wrapped in scaffolding that takes tool actions), elicitation in the question, and headroom are unexplained, and none of them is in the definitions list. Readers outside the field may not grasp what was improved or measured. Smallest repair: add one clause each saying what an agent is and what post-training is, and suggest term-library entries for these terms.
Our agents consist of a large language model wrapped by an “agent scaffolding” program that lets the model repeatedly decide to take an action using a tool
date — pass: The reading gives March 15, 2024, which matches the originalPublished instant's UTC day and the date in the URL. It adds no hindsight framing or later events. The source is a publisher page, so no archive disclosure is required.
support — pass: Every claim matches the source text: the safety-margin protocol, 5±2% to 30±6%, 30±6% to 38±6% (not significant), the task suite not targeting dangerous capabilities, results inconclusive with headroom expected, and estimates that could be too high or too low. The question follows from METR's own stated limitation and is not leading.
It is plausible that more threat-model-relevant capabilities might be further from the tasks that OpenAI has already optimized for, so there may be more headroom left for easy improvements.
media — pass: No media was selected. No verified media candidates were supplied, so nothing is misidentified.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRA test of whether a result is unlikely to be due to chance alone; it does not say how large or important an effect is.
Definition referenceFurther training of an existing model to adapt it to a particular task or use.
Definition referenceMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRA test of whether a result is unlikely to be due to chance alone; it does not say how large or important an effect is.
Definition referenceFurther training of an existing model to adapt it to a particular task or use.
Definition referenceMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference