{"id":"d1e07fc8487f58e81e01393401e48b7abbd918611c598a9b63c11d8e5d684b06","title":"METR describes its work in its year-in-review","argument":"METR published this year-in-review on February 7, 2024, describing work during its first full year of operation. METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks, and that its testing formed part of the OpenAI system card for GPT-4. It also says its work was presented at the UK’s AI Safety Summit, while the White House Executive Order on AI required developers to share certain red-team testing results related to self-replication or propagation.\n\nMETR argues that Responsible Scaling Policies (RSPs)—a framework it proposed for planning responses to evidence of danger from AI system evaluations—gained traction. The post points to recommendations by AI researchers and Eric Schmidt, and to Anthropic and OpenAI releasing their own policies or frameworks. METR also reports starting a partnership with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team. The post presents these developments as progress in evaluation and safety planning; it does not report whether the evaluations identify danger reliably or what consequences they have had in practice.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 30, 2026; originally published February 7, 2024. The post does not define “LM agents” or report evidence about the reliability or practical effects of the evaluations. The post is METR’s account of its own work and reported developments; it provides no evaluation results in this text.","speaker":{"name":"METR","roleAtSource":"Publisher"},"source":{"title":"2023 Year In Review","publisher":"METR","format":"Article","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","publishedAt":"2024-02-07T08:00:00.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks.","excerpt":"We developed our initial methodology for evaluating LM agents on realistic autonomous tasks","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","relationship":"Source-stated"},{"text":"METR says its testing formed part of the OpenAI system card for GPT-4.","excerpt":"Our testing formed part of the OpenAI system card for the release of GPT-4.","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","relationship":"Source-stated"},{"text":"METR says the White House Executive Order on AI required developers to share results of certain red-team testing related to self-replication or propagation.","excerpt":"The White House Executive Order on AI stated a requirement that AI developers share “the results of any red-team testing that the company has conducted relating to… the possibility for self-replication or propagation” of AI systems.","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","relationship":"Source-stated"},{"text":"METR says it proposed RSPs as a framework for planning responses to evidence of danger from AI system evaluations, and that the framework gained substantial traction.","excerpt":"We proposed “Responsible Scaling Policies” (RSPs), a framework for planning response to evidence of danger from AI system evaluations that AI developers can adopt unilaterally. This framework has gained substantial traction throughout the industry and AI policy community.","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","relationship":"Source-stated"},{"text":"METR says it started partnering with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team.","excerpt":"We started partnering with the UK AI Safety Institute (AISI) (formerly the Frontier AI Taskforce) to develop evaluations for their Autonomous Systems team.","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","relationship":"Source-stated"},{"text":"Escape Meridian reads the post as presenting evaluation and safety-planning work as progress, while leaving its practical effectiveness unreported.","excerpt":"We developed our initial methodology for evaluating LM agents on realistic autonomous tasks","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://metr.org/blog/2024-02-07-2023-year-in-review/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What evidence will show whether these evaluations can identify dangerous autonomous capabilities reliably?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"METR describes its work in its year-in-review","summary":"METR published this year-in-review on February 7, 2024, describing work during its first full year of operation. METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks, and that its testing formed part of the OpenAI system card for GPT-4. It also says its work was presented at the UK’s AI Safety Summit, while the White House Executive Order on AI required developers to share certain red-team testing results related to self-replication or propagation.\n\nMETR argues that Responsible Scaling Policies (RSPs)—a framework it proposed for planning responses to evidence of danger from AI system evaluations—gained traction. The post points to recommendations by AI researchers and Eric Schmidt, and to Anthropic and OpenAI releasing their own policies or frameworks. METR also reports starting a partnership with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team. The post presents these developments as progress in evaluation and safety planning; it does not report whether the evaluations identify danger reliably or what consequences they have had in practice.","question":"What evidence will show whether these evaluations can identify dangerous autonomous capabilities reliably?","claims":[{"text":"METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks.","excerpt":"We developed our initial methodology for evaluating LM agents on realistic autonomous tasks","support":"source-stated"},{"text":"METR says its testing formed part of the OpenAI system card for GPT-4.","excerpt":"Our testing formed part of the OpenAI system card for the release of GPT-4.","support":"source-stated"},{"text":"METR says the White House Executive Order on AI required developers to share results of certain red-team testing related to self-replication or propagation.","excerpt":"The White House Executive Order on AI stated a requirement that AI developers share “the results of any red-team testing that the company has conducted relating to… the possibility for self-replication or propagation” of AI systems.","support":"source-stated"},{"text":"METR says it proposed RSPs as a framework for planning responses to evidence of danger from AI system evaluations, and that the framework gained substantial traction.","excerpt":"We proposed “Responsible Scaling Policies” (RSPs), a framework for planning response to evidence of danger from AI system evaluations that AI developers can adopt unilaterally. This framework has gained substantial traction throughout the industry and AI policy community.","support":"source-stated"},{"text":"METR says it started partnering with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team.","excerpt":"We started partnering with the UK AI Safety Institute (AISI) (formerly the Frontier AI Taskforce) to develop evaluations for their Autonomous Systems team.","support":"source-stated"},{"text":"Escape Meridian reads the post as presenting evaluation and safety-planning work as progress, while leaving its practical effectiveness unreported.","excerpt":"We developed our initial methodology for evaluating LM agents on realistic autonomous tasks","support":"interpretation"}],"limitations":["Text as retrieved on October 30, 2026; originally published February 7, 2024.","The post does not define “LM agents” or report evidence about the reliability or practical effects of the evaluations.","The post is METR’s account of its own work and reported developments; it provides no evaluation results in this text."],"mediaId":null,"correctionReason":null,"id":"d1e07fc8487f58e81e01393401e48b7abbd918611c598a9b63c11d8e5d684b06","publicationId":"54051d47-54e7-e9c9-249d-8db1c305ecd2","version":1,"sourceUrl":"https://metr.org/blog/2024-02-07-2023-year-in-review/","sourceTitle":"2023 Year In Review","publisher":"METR","publishedAt":"2024-02-07T08:00:00.000Z","originalPublished":{"value":"2024-02-07T08:00:00.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T03:57:18.563Z","createdAt":"2026-11-21T16:48:13.278Z","historical":true,"entityIds":["metr"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://metr.org/blog/2024-02-07-2023-year-in-review/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"7784e07461399165c1d24fbe63937873c6776d1e2d1485587c207d2f3ea7b5b0","rubricVersion":"unrecorded","provider":"anthropic","model":"claude-opus-5-5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-30T02:26:13.379Z"},"provenance":{"textHash":"dc6f956926e5ad2ba9078c179e72d0efedd4bf19e3355064c87ff279ec303b1a","observationId":"781a9061e34c2e3635be918441f655a3394697629ba333e1d1274beb486bc3b6","retrievedAt":"2026-10-30T02:27:47.465Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"8dad34d6-d022-dfba-97b2-b23f651d14b2"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"8390a8f0-b492-9dc1-fc4f-abf4f26663d3"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-21T16:43:26.567Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/metr-describes-its-work-in-its-year-in-review-d1e07fc8","formats":{"html":"/readings/metr-describes-its-work-in-its-year-in-review-d1e07fc8","markdown":"/readings/metr-describes-its-work-in-its-year-in-review-d1e07fc8.md","json":"/readings/metr-describes-its-work-in-its-year-in-review-d1e07fc8.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-red-teaming","name":"Red-teaming","explanation":"Deliberately attacking or tricking a system to find its weaknesses before others do.","referenceUrl":"https://en.wikipedia.org/wiki/Red_team"}],"entities":[{"id":"metr","name":"METR","kind":"organization","context":"METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.","referenceUrls":["https://metr.org/"],"path":"/entities/metr","json":"/entities/metr.json","markdown":"/entities/metr.md"},{"id":"openai","name":"OpenAI","kind":"organization","context":"OpenAI develops AI models and products and publishes research, release announcements and evaluations.","referenceUrls":["https://openai.com/news/"],"path":"/entities/openai","json":"/entities/openai.json","markdown":"/entities/openai.md"},{"id":"anthropic","name":"Anthropic","kind":"organization","context":"Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems.","referenceUrls":["https://www.anthropic.com/news","https://www.anthropic.com/research/team/economics"],"path":"/entities/anthropic","json":"/entities/anthropic.json","markdown":"/entities/anthropic.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"d1e07fc8487f58e81e01393401e48b7abbd918611c598a9b63c11d8e5d684b06","revision":"54051d47-54e7-e9c9-249d-8db1c305ecd2","revisionKind":"publication-id","title":"METR describes its work in its year-in-review","url":"https://escapemeridian.com/readings/metr-describes-its-work-in-its-year-in-review-d1e07fc8"},"source":{"type":"source","id":"https://metr.org/blog/2024-02-07-2023-year-in-review/","revision":"dc6f956926e5ad2ba9078c179e72d0efedd4bf19e3355064c87ff279ec303b1a","revisionKind":"sha256-source-text","title":"2023 Year In Review","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter."},"revisions":[{"id":"d1e07fc8487f58e81e01393401e48b7abbd918611c598a9b63c11d8e5d684b06","title":"METR describes its work in its year-in-review","argument":"METR published this year-in-review on February 7, 2024, describing work during its first full year of operation. METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks, and that its testing formed part of the OpenAI system card for GPT-4. It also says its work was presented at the UK’s AI Safety Summit, while the White House Executive Order on AI required developers to share certain red-team testing results related to self-replication or propagation.\n\nMETR argues that Responsible Scaling Policies (RSPs)—a framework it proposed for planning responses to evidence of danger from AI system evaluations—gained traction. The post points to recommendations by AI researchers and Eric Schmidt, and to Anthropic and OpenAI releasing their own policies or frameworks. METR also reports starting a partnership with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team. The post presents these developments as progress in evaluation and safety planning; it does not report whether the evaluations identify danger reliably or what consequences they have had in practice.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 30, 2026; originally published February 7, 2024. The post does not define “LM agents” or report evidence about the reliability or practical effects of the evaluations. The post is METR’s account of its own work and reported developments; it provides no evaluation results in this text.","speaker":{"name":"METR","roleAtSource":"Publisher"},"source":{"title":"2023 Year In Review","publisher":"METR","format":"Article","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","publishedAt":"2024-02-07T08:00:00.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks.","excerpt":"We developed our initial methodology for evaluating LM agents on realistic autonomous tasks","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","relationship":"Source-stated"},{"text":"METR says its testing formed part of the OpenAI system card for GPT-4.","excerpt":"Our testing formed part of the OpenAI system card for the release of GPT-4.","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","relationship":"Source-stated"},{"text":"METR says the White House Executive Order on AI required developers to share results of certain red-team testing related to self-replication or propagation.","excerpt":"The White House Executive Order on AI stated a requirement that AI developers share “the results of any red-team testing that the company has conducted relating to… the possibility for self-replication or propagation” of AI systems.","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","relationship":"Source-stated"},{"text":"METR says it proposed RSPs as a framework for planning responses to evidence of danger from AI system evaluations, and that the framework gained substantial traction.","excerpt":"We proposed “Responsible Scaling Policies” (RSPs), a framework for planning response to evidence of danger from AI system evaluations that AI developers can adopt unilaterally. This framework has gained substantial traction throughout the industry and AI policy community.","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","relationship":"Source-stated"},{"text":"METR says it started partnering with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team.","excerpt":"We started partnering with the UK AI Safety Institute (AISI) (formerly the Frontier AI Taskforce) to develop evaluations for their Autonomous Systems team.","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","relationship":"Source-stated"},{"text":"Escape Meridian reads the post as presenting evaluation and safety-planning work as progress, while leaving its practical effectiveness unreported.","excerpt":"We developed our initial methodology for evaluating LM agents on realistic autonomous tasks","url":"https://metr.org/blog/2024-02-07-2023-year-in-review/","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://metr.org/blog/2024-02-07-2023-year-in-review/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What evidence will show whether these evaluations can identify dangerous autonomous capabilities reliably?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"METR describes its work in its year-in-review","summary":"METR published this year-in-review on February 7, 2024, describing work during its first full year of operation. METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks, and that its testing formed part of the OpenAI system card for GPT-4. It also says its work was presented at the UK’s AI Safety Summit, while the White House Executive Order on AI required developers to share certain red-team testing results related to self-replication or propagation.\n\nMETR argues that Responsible Scaling Policies (RSPs)—a framework it proposed for planning responses to evidence of danger from AI system evaluations—gained traction. The post points to recommendations by AI researchers and Eric Schmidt, and to Anthropic and OpenAI releasing their own policies or frameworks. METR also reports starting a partnership with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team. The post presents these developments as progress in evaluation and safety planning; it does not report whether the evaluations identify danger reliably or what consequences they have had in practice.","question":"What evidence will show whether these evaluations can identify dangerous autonomous capabilities reliably?","claims":[{"text":"METR says it developed an initial methodology for evaluating LM agents on realistic autonomous tasks.","excerpt":"We developed our initial methodology for evaluating LM agents on realistic autonomous tasks","support":"source-stated"},{"text":"METR says its testing formed part of the OpenAI system card for GPT-4.","excerpt":"Our testing formed part of the OpenAI system card for the release of GPT-4.","support":"source-stated"},{"text":"METR says the White House Executive Order on AI required developers to share results of certain red-team testing related to self-replication or propagation.","excerpt":"The White House Executive Order on AI stated a requirement that AI developers share “the results of any red-team testing that the company has conducted relating to… the possibility for self-replication or propagation” of AI systems.","support":"source-stated"},{"text":"METR says it proposed RSPs as a framework for planning responses to evidence of danger from AI system evaluations, and that the framework gained substantial traction.","excerpt":"We proposed “Responsible Scaling Policies” (RSPs), a framework for planning response to evidence of danger from AI system evaluations that AI developers can adopt unilaterally. This framework has gained substantial traction throughout the industry and AI policy community.","support":"source-stated"},{"text":"METR says it started partnering with the UK AI Safety Institute to develop evaluations for its Autonomous Systems team.","excerpt":"We started partnering with the UK AI Safety Institute (AISI) (formerly the Frontier AI Taskforce) to develop evaluations for their Autonomous Systems team.","support":"source-stated"},{"text":"Escape Meridian reads the post as presenting evaluation and safety-planning work as progress, while leaving its practical effectiveness unreported.","excerpt":"We developed our initial methodology for evaluating LM agents on realistic autonomous tasks","support":"interpretation"}],"limitations":["Text as retrieved on October 30, 2026; originally published February 7, 2024.","The post does not define “LM agents” or report evidence about the reliability or practical effects of the evaluations.","The post is METR’s account of its own work and reported developments; it provides no evaluation results in this text."],"mediaId":null,"correctionReason":null,"id":"d1e07fc8487f58e81e01393401e48b7abbd918611c598a9b63c11d8e5d684b06","publicationId":"54051d47-54e7-e9c9-249d-8db1c305ecd2","version":1,"sourceUrl":"https://metr.org/blog/2024-02-07-2023-year-in-review/","sourceTitle":"2023 Year In Review","publisher":"METR","publishedAt":"2024-02-07T08:00:00.000Z","originalPublished":{"value":"2024-02-07T08:00:00.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T03:57:18.563Z","createdAt":"2026-11-21T16:48:13.278Z","historical":true,"entityIds":["metr"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://metr.org/blog/2024-02-07-2023-year-in-review/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"7784e07461399165c1d24fbe63937873c6776d1e2d1485587c207d2f3ea7b5b0","rubricVersion":"unrecorded","provider":"anthropic","model":"claude-opus-5-5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-30T02:26:13.379Z"},"provenance":{"textHash":"dc6f956926e5ad2ba9078c179e72d0efedd4bf19e3355064c87ff279ec303b1a","observationId":"781a9061e34c2e3635be918441f655a3394697629ba333e1d1274beb486bc3b6","retrievedAt":"2026-10-30T02:27:47.465Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"8dad34d6-d022-dfba-97b2-b23f651d14b2"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"8390a8f0-b492-9dc1-fc4f-abf4f26663d3"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-21T16:43:26.567Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"54051d47-54e7-e9c9-249d-8db1c305ecd2","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"This is a single METR blog post giving its own year-in-review account, not a listing. The reading attributes each point to METR ('METR says', 'METR argues', 'The post points to'), including the third-party developments METR reports, such as the Executive Order, the RSP endorsements and the Anthropic and OpenAI frameworks.","excerpt":"Here's a summary of what METR accomplished in 2023 -- our first full year of operation."},{"dimension":"summary","status":"uncertain","explanation":"The reading leads with the term 'LM agents' but never explains it, and it is not in the definitions list, which defines only red-teaming. A non-specialist may not know this means AI language models acting as agents that carry out tasks on their own, yet it is central to METR's main methodology claim. Smallest repair: add a plain-language gloss or a term-library entry for 'LM agents'. A smaller gap: the reading drops METR's framing that 'the idea of doing autonomous replication testing gained adoption', which links the Executive Order and GPT-4 items to METR's work. The rest is faithful, and the title and question are sound.","excerpt":"We developed our initial methodology for evaluating LM agents on realistic autonomous tasks"},{"dimension":"date","status":"pass","explanation":"evidence.originalPublished is 2024-02-07T08:00Z, so the UTC day is February 7, 2024, matching the reading. The source URL path also shows 2024-02-07. The kind is publisher-page, so no archive disclosure is needed. The reading presents no hindsight framing and no later events, and it states no dates finer than the source's.","excerpt":null},{"dimension":"support","status":"pass","explanation":"Each claim matches its verbatim excerpt. The Escape Meridian interpretation is labeled as such and only says the post leaves practical effectiveness unreported, which is consistent with the text. The question asks about a real gap the evidence leaves open rather than asserting an answer. No benefits or costs are invented.","excerpt":"This framework has gained substantial traction throughout the industry and AI policy community."},{"dimension":"media","status":"pass","explanation":"No media candidates were supplied, sourceMedia items are empty, and the reading selects none, so this is consistent.","excerpt":null}],"createdAt":"2026-11-21T16:43:26.567Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}