· Feb 7, 2024
METR describes its work in its year-in-review
Markdown · JSON · Original source
published this year-in-review on February 7, 2024, describing work during its first full year of operation. METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks, and that its testing formed part of the system card for GPT-4. It also says its work was presented at the UK’s AI Safety Summit, while the White House Executive Order on AI required developers to share certain testing results related to self-replication or propagation. METR argues that Responsible Scaling Policies (RSPs)—a framework it proposed for planning responses to evidence of danger from AI system evaluations—gained traction. The post points to recommendations by AI researchers and Eric Schmidt, and to and OpenAI releasing their own policies or frameworks. METR also reports starting a partnership with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team. The post presents these developments as progress in evaluation and safety planning; it does not report whether the evaluations identify danger reliably or what consequences they have had in practice.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
· Article
Source details
Original source date: Feb 7, 2024 (2024-02-07T08:00:00.000Z).
Date basis: Date supplied by the publisher’s feed.
Added to Escape Meridian: 2026-11-21T16:48:13.278Z. First observed: 2026-10-18T03:57:18.563Z. Retrieved: 2026-10-30T02:27:47.465Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
Text as retrieved on October 30, 2026; originally published February 7, 2024. The post does not define “LM agents” or report evidence about the reliability or practical effects of the evaluations. The post is ’s account of its own work and reported developments; it provides no evaluation results in this text.
Open question
What evidence will show whether these evaluations can identify dangerous autonomous capabilities reliably?
Source-stated
METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks.
We developed our initial methodology for evaluating LM agents on realistic autonomous tasks
Source-stated
METR says its testing formed part of the OpenAI system card for GPT-4.
Our testing formed part of the OpenAI system card for the release of GPT-4.
Source-stated
METR says the White House Executive Order on AI required developers to share results of certain red-team testing related to self-replication or propagation.
The White House Executive Order on AI stated a requirement that AI developers share “the results of any red-team testing that the company has conducted relating to… the possibility for self-replication or propagation” of AI systems.
Source-stated
METR says it proposed RSPs as a framework for planning responses to evidence of danger from AI system evaluations, and that the framework gained substantial traction.
We proposed “Responsible Scaling Policies” (RSPs), a framework for planning response to evidence of danger from AI system evaluations that AI developers can adopt unilaterally. This framework has gained substantial traction throughout the industry and AI policy community.
Source-stated
METR says it started partnering with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team.
We started partnering with the UK AI Safety Institute (AISI) (formerly the Frontier AI Taskforce) to develop evaluations for their Autonomous Systems team.
Escape Meridian interpretation
Escape Meridian reads the post as presenting evaluation and safety-planning work as progress, while leaving its practical effectiveness unreported.
We developed our initial methodology for evaluating LM agents on realistic autonomous tasks
Source and contributor context
METR · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Red-teaming
Deliberately attacking or tricking a system to find its weaknesses before others do.
METR
METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
OpenAI
OpenAI develops AI models and products and publishes research, release announcements and evaluations.
Anthropic
Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Editorial provenance
Version 1. Publication ID: 54051d47-54e7-e9c9-249d-8db1c305ecd2.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-21T16:43:26.567Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.
Observation: 781a9061e34c2e3635be918441f655a3394697629ba333e1d1274beb486bc3b6. Source text hash: dc6f956926e5ad2ba9078c179e72d0efedd4bf19e3355064c87ff279ec303b1a.
Revision and source-check history
Version 1 · 2026-11-21T16:48:13.278Z
METR describes its work in its year-in-review
published this year-in-review on February 7, 2024, describing work during its first full year of operation. METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks, and that its testing formed part of the system card for GPT-4. It also says its work was presented at the UK’s AI Safety Summit, while the White House Executive Order on AI required developers to share certain testing results related to self-replication or propagation. METR argues that Responsible Scaling Policies (RSPs)—a framework it proposed for planning responses to evidence of danger from AI system evaluations—gained traction. The post points to recommendations by AI researchers and Eric Schmidt, and to and OpenAI releasing their own policies or frameworks. METR also reports starting a partnership with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team. The post presents these developments as progress in evaluation and safety planning; it does not report whether the evaluations identify danger reliably or what consequences they have had in practice.
Original source
· Article
Source details
Original source date: Feb 7, 2024 (2024-02-07T08:00:00.000Z).
Date basis: Date supplied by the publisher’s feed.
Added to Escape Meridian: 2026-11-21T16:48:13.278Z. First observed: 2026-10-18T03:57:18.563Z. Retrieved: 2026-10-30T02:27:47.465Z.
Limits
Text as retrieved on October 30, 2026; originally published February 7, 2024. The post does not define “LM agents” or report evidence about the reliability or practical effects of the evaluations. The post is ’s account of its own work and reported developments; it provides no evaluation results in this text.
Open question
What evidence will show whether these evaluations can identify dangerous autonomous capabilities reliably?
Source-stated
METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks.
We developed our initial methodology for evaluating LM agents on realistic autonomous tasks
Source-stated
METR says its testing formed part of the OpenAI system card for GPT-4.
Our testing formed part of the OpenAI system card for the release of GPT-4.
Source-stated
METR says the White House Executive Order on AI required developers to share results of certain red-team testing related to self-replication or propagation.
The White House Executive Order on AI stated a requirement that AI developers share “the results of any red-team testing that the company has conducted relating to… the possibility for self-replication or propagation” of AI systems.
Source-stated
METR says it proposed RSPs as a framework for planning responses to evidence of danger from AI system evaluations, and that the framework gained substantial traction.
We proposed “Responsible Scaling Policies” (RSPs), a framework for planning response to evidence of danger from AI system evaluations that AI developers can adopt unilaterally. This framework has gained substantial traction throughout the industry and AI policy community.
Source-stated
METR says it started partnering with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team.
We started partnering with the UK AI Safety Institute (AISI) (formerly the Frontier AI Taskforce) to develop evaluations for their Autonomous Systems team.
Escape Meridian interpretation
Escape Meridian reads the post as presenting evaluation and safety-planning work as progress, while leaving its practical effectiveness unreported.
We developed our initial methodology for evaluating LM agents on realistic autonomous tasks
Source and contributor context
METR · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Red-teaming
Deliberately attacking or tricking a system to find its weaknesses before others do.
METR
METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
OpenAI
OpenAI develops AI models and products and publishes research, release announcements and evaluations.
Anthropic
Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Editorial provenance
Version 1. Publication ID: 54051d47-54e7-e9c9-249d-8db1c305ecd2.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-21T16:43:26.567Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.
Observation: 781a9061e34c2e3635be918441f655a3394697629ba333e1d1274beb486bc3b6. Source text hash: dc6f956926e5ad2ba9078c179e72d0efedd4bf19e3355064c87ff279ec303b1a.
uncertain · 2026-11-21T16:43:26.567Z · anthropic/claude-opus-5.5
Publication: 54051d47-54e7-e9c9-249d-8db1c305ecd2
attribution — pass: This is a single METR blog post giving its own year-in-review account, not a listing. The reading attributes each point to METR ('METR says', 'METR argues', 'The post points to'), including the third-party developments METR reports, such as the Executive Order, the RSP endorsements and the Anthropic and OpenAI frameworks.
Here's a summary of what METR accomplished in 2023 -- our first full year of operation.
summary — uncertain: The reading leads with the term 'LM agents' but never explains it, and it is not in the definitions list, which defines only red-teaming. A non-specialist may not know this means AI language models acting as agents that carry out tasks on their own, yet it is central to METR's main methodology claim. Smallest repair: add a plain-language gloss or a term-library entry for 'LM agents'. A smaller gap: the reading drops METR's framing that 'the idea of doing autonomous replication testing gained adoption', which links the Executive Order and GPT-4 items to METR's work. The rest is faithful, and the title and question are sound.
We developed our initial methodology for evaluating LM agents on realistic autonomous tasks
date — pass: evidence.originalPublished is 2024-02-07T08:00Z, so the UTC day is February 7, 2024, matching the reading. The source URL path also shows 2024-02-07. The kind is publisher-page, so no archive disclosure is needed. The reading presents no hindsight framing and no later events, and it states no dates finer than the source's.
support — pass: Each claim matches its verbatim excerpt. The Escape Meridian interpretation is labeled as such and only says the post leaves practical effectiveness unreported, which is consistent with the text. The question asks about a real gap the evidence leaves open rather than asserting an answer. No benefits or costs are invented.
This framework has gained substantial traction throughout the industry and AI policy community.
media — pass: No media candidates were supplied, sourceMedia items are empty, and the reading selects none, so this is consistent.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METROpenAI develops AI models and products and publishes research, release announcements and evaluations.
Explore OpenAIDeliberately attacking or tricking a system to find its weaknesses before others do.
Definition referenceAnthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Explore AnthropicMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METROpenAI develops AI models and products and publishes research, release announcements and evaluations.
Explore OpenAIDeliberately attacking or tricking a system to find its weaknesses before others do.
Definition referenceAnthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Explore AnthropicMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METR