Escape Meridian

· Feb 7, 2024

METR describes its work in its year-in-review

Markdown · JSON · Original source

published this year-in-review on February 7, 2024, describing work during its first full year of operation. METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks, and that its testing formed part of the system card for GPT-4. It also says its work was presented at the UK’s AI Safety Summit, while the White House Executive Order on AI required developers to share certain testing results related to self-replication or propagation. METR argues that Responsible Scaling Policies (RSPs)—a framework it proposed for planning responses to evidence of danger from AI system evaluations—gained traction. The post points to recommendations by AI researchers and Eric Schmidt, and to and OpenAI releasing their own policies or frameworks. METR also reports starting a partnership with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team. The post presents these developments as progress in evaluation and safety planning; it does not report whether the evaluations identify danger reliably or what consequences they have had in practice.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

METRArticle · metr.orgOpen original source

Original source

· Article

Source details

Original source date: Feb 7, 2024 (2024-02-07T08:00:00.000Z).

Date basis: Date supplied by the publisher’s feed.

Added to Escape Meridian: 2026-11-21T16:48:13.278Z. First observed: 2026-10-18T03:57:18.563Z. Retrieved: 2026-10-30T02:27:47.465Z.

Referenced by

Recorded citations in public readings. A reference alone does not establish agreement or influence.

No public references recorded here yet.

Reference data (JSON)

Limits

Text as retrieved on October 30, 2026; originally published February 7, 2024. The post does not define “LM agents” or report evidence about the reliability or practical effects of the evaluations. The post is ’s account of its own work and reported developments; it provides no evaluation results in this text.

Open question

What evidence will show whether these evaluations can identify dangerous autonomous capabilities reliably?

Source-stated

METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks.

We developed our initial methodology for evaluating LM agents on realistic autonomous tasks

Source-stated

METR says its testing formed part of the OpenAI system card for GPT-4.

Our testing formed part of the OpenAI system card for the release of GPT-4.

Source-stated

METR says the White House Executive Order on AI required developers to share results of certain red-team testing related to self-replication or propagation.

The White House Executive Order on AI stated a requirement that AI developers share “the results of any red-team testing that the company has conducted relating to… the possibility for self-replication or propagation” of AI systems.

Source-stated

METR says it proposed RSPs as a framework for planning responses to evidence of danger from AI system evaluations, and that the framework gained substantial traction.

We proposed “Responsible Scaling Policies” (RSPs), a framework for planning response to evidence of danger from AI system evaluations that AI developers can adopt unilaterally. This framework has gained substantial traction throughout the industry and AI policy community.

Source-stated

METR says it started partnering with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team.

We started partnering with the UK AI Safety Institute (AISI) (formerly the Frontier AI Taskforce) to develop evaluations for their Autonomous Systems team.

Escape Meridian interpretation

Escape Meridian reads the post as presenting evaluation and safety-planning work as progress, while leaving its practical effectiveness unreported.

We developed our initial methodology for evaluating LM agents on realistic autonomous tasks

Source and contributor context

METR · Publisher

The publisher is identified here. No individual author has been inferred.

Source media

Media retrieval status: unchecked.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Red-teaming

Deliberately attacking or tricking a system to find its weaknesses before others do.

METR

METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.

OpenAI

OpenAI develops AI models and products and publishes research, release announcements and evaluations.

Anthropic

Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems.

Editorial provenance

Version 1. Publication ID: 54051d47-54e7-e9c9-249d-8db1c305ecd2.

Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-21T16:43:26.567Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.

Observation: 781a9061e34c2e3635be918441f655a3394697629ba333e1d1274beb486bc3b6. Source text hash: dc6f956926e5ad2ba9078c179e72d0efedd4bf19e3355064c87ff279ec303b1a.

Revision and source-check history

Version 1 · 2026-11-21T16:48:13.278Z

METR describes its work in its year-in-review

published this year-in-review on February 7, 2024, describing work during its first full year of operation. METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks, and that its testing formed part of the system card for GPT-4. It also says its work was presented at the UK’s AI Safety Summit, while the White House Executive Order on AI required developers to share certain testing results related to self-replication or propagation. METR argues that Responsible Scaling Policies (RSPs)—a framework it proposed for planning responses to evidence of danger from AI system evaluations—gained traction. The post points to recommendations by AI researchers and Eric Schmidt, and to and OpenAI releasing their own policies or frameworks. METR also reports starting a partnership with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team. The post presents these developments as progress in evaluation and safety planning; it does not report whether the evaluations identify danger reliably or what consequences they have had in practice.

Original source

· Article

Source details

Original source date: Feb 7, 2024 (2024-02-07T08:00:00.000Z).

Date basis: Date supplied by the publisher’s feed.

Added to Escape Meridian: 2026-11-21T16:48:13.278Z. First observed: 2026-10-18T03:57:18.563Z. Retrieved: 2026-10-30T02:27:47.465Z.

Limits

Text as retrieved on October 30, 2026; originally published February 7, 2024. The post does not define “LM agents” or report evidence about the reliability or practical effects of the evaluations. The post is ’s account of its own work and reported developments; it provides no evaluation results in this text.

Open question

What evidence will show whether these evaluations can identify dangerous autonomous capabilities reliably?

Source-stated

METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks.

We developed our initial methodology for evaluating LM agents on realistic autonomous tasks

Source-stated

METR says its testing formed part of the OpenAI system card for GPT-4.

Our testing formed part of the OpenAI system card for the release of GPT-4.

Source-stated

METR says the White House Executive Order on AI required developers to share results of certain red-team testing related to self-replication or propagation.

The White House Executive Order on AI stated a requirement that AI developers share “the results of any red-team testing that the company has conducted relating to… the possibility for self-replication or propagation” of AI systems.

Source-stated

METR says it proposed RSPs as a framework for planning responses to evidence of danger from AI system evaluations, and that the framework gained substantial traction.

We proposed “Responsible Scaling Policies” (RSPs), a framework for planning response to evidence of danger from AI system evaluations that AI developers can adopt unilaterally. This framework has gained substantial traction throughout the industry and AI policy community.

Source-stated

METR says it started partnering with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team.

We started partnering with the UK AI Safety Institute (AISI) (formerly the Frontier AI Taskforce) to develop evaluations for their Autonomous Systems team.

Escape Meridian interpretation

Escape Meridian reads the post as presenting evaluation and safety-planning work as progress, while leaving its practical effectiveness unreported.

We developed our initial methodology for evaluating LM agents on realistic autonomous tasks

Source and contributor context

METR · Publisher

The publisher is identified here. No individual author has been inferred.

Source media

Media retrieval status: unchecked.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Red-teaming

Deliberately attacking or tricking a system to find its weaknesses before others do.

METR

METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.

OpenAI

OpenAI develops AI models and products and publishes research, release announcements and evaluations.

Anthropic

Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems.

Editorial provenance

Version 1. Publication ID: 54051d47-54e7-e9c9-249d-8db1c305ecd2.

Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-21T16:43:26.567Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.

Observation: 781a9061e34c2e3635be918441f655a3394697629ba333e1d1274beb486bc3b6. Source text hash: dc6f956926e5ad2ba9078c179e72d0efedd4bf19e3355064c87ff279ec303b1a.

uncertain · 2026-11-21T16:43:26.567Z · anthropic/claude-opus-5.5

Publication: 54051d47-54e7-e9c9-249d-8db1c305ecd2

attribution — pass: This is a single METR blog post giving its own year-in-review account, not a listing. The reading attributes each point to METR ('METR says', 'METR argues', 'The post points to'), including the third-party developments METR reports, such as the Executive Order, the RSP endorsements and the Anthropic and OpenAI frameworks.

Here's a summary of what METR accomplished in 2023 -- our first full year of operation.

summary — uncertain: The reading leads with the term 'LM agents' but never explains it, and it is not in the definitions list, which defines only red-teaming. A non-specialist may not know this means AI language models acting as agents that carry out tasks on their own, yet it is central to METR's main methodology claim. Smallest repair: add a plain-language gloss or a term-library entry for 'LM agents'. A smaller gap: the reading drops METR's framing that 'the idea of doing autonomous replication testing gained adoption', which links the Executive Order and GPT-4 items to METR's work. The rest is faithful, and the title and question are sound.

We developed our initial methodology for evaluating LM agents on realistic autonomous tasks

date — pass: evidence.originalPublished is 2024-02-07T08:00Z, so the UTC day is February 7, 2024, matching the reading. The source URL path also shows 2024-02-07. The kind is publisher-page, so no archive disclosure is needed. The reading presents no hindsight framing and no later events, and it states no dates finer than the source's.

support — pass: Each claim matches its verbatim excerpt. The Escape Meridian interpretation is labeled as such and only says the post leaves practical effectiveness unreported, which is consistent with the text. The question asks about a real gap the evidence leaves open rather than asserting an answer. No benefits or costs are invented.

This framework has gained substantial traction throughout the industry and AI policy community.

media — pass: No media candidates were supplied, sourceMedia items are empty, and the reading selects none, so this is consistent.

Challenge or correct this reading

Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.