# METR reports Claude Code and Codex make little difference in measured time horizon [Canonical reading](https://escapemeridian.com/readings/metr-reports-claude-code-and-codex-make-little-difference-in-a57da445) METR’s note, published February 13, 2026, reports that Claude Code and Codex were not obviously better than METR’s default scaffolds when measuring Opus 4.5 and GPT-5. The author concludes that these specialized setups “doesn’t make a huge difference” to the models’ measured time horizon, and thinks the difference probably is not huge for other recent models either. The author points to a key difference in how the setups are used: Claude Code and Codex are mostly used interactively, with a human checking the agent’s work, while METR’s tasks are autonomous. The task suite may also differ from real-world work. This leaves open whether the comparison would look different with human check-ins or a different task mix. The note’s examples include GPT-5 with Codex addressing a nonexistent user, and Opus 4.5 with Claude Code sticking with a poor plan or wasting limited attempts without using results. These examples describe observed runs, not how often such failures occur elsewhere. The author notes possible effects from token budgets, rough wrappers and technical issues; a higher token budget barely changed a preliminary measurement. The note’s conclusion is limited to the tested models and task setting. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source METR · Article Original source date: Feb 13, 2026 (2026-02-13T08:00:00.000Z). Date basis: Date supplied by the publisher’s feed. Added to Escape Meridian: 2026-11-08T05:23:23.785Z. First observed: 2026-10-18T03:09:19.726Z. Retrieved: 2026-10-23T21:09:37.865Z. - [Measuring Time Horizon using Claude Code and Codex](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Limits Text as retrieved on October 23, 2026; originally published February 13, 2026. The comparison concerns Opus 4.5 and GPT-5 on METR’s task suite; the author says the real-world distribution of task types may differ. The author notes possible effects from token budgets, rough wrappers and minor technical issues. The note describes the findings as unpolished research updates that have gone through less review than METR’s research posts. The qualitative failure examples do not establish how often those behaviors occur outside the reported runs. ## Open question How would the comparison change if Claude Code and Codex were evaluated in the interactive settings where a human checks the agent’s work? ## Source-stated METR’s note reports that neither specialized scaffold was obviously better than METR’s default scaffolds for the tested models. > It looks like Claude Code and Codex aren’t obviously better than the default scaffolds we use for our agents. - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source-stated The author concludes that using Codex and Claude Code does not make a large difference to the measured time horizon of GPT-5 and Opus 4.5. > for GPT-5 and Opus 4.5, using Codex and Claude Code doesn’t make a huge difference for the time horizon of the model. - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source-stated The author says neither model’s difference was statistically significant in the comparison. > Neither the GPT-5 time horizon difference, nor the Opus 4.5 time horizon difference, are statistically significant: - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source-stated The author identifies the difference between interactive use and METR’s autonomous tasks as one reason the specialized scaffolds may not outperform the defaults in this evaluation. > Those scaffolds are mostly used in an interactive setting, where a human user often checks in on the agent’s work. In contrast, agents perform our time horizon tasks autonomously, without any human input. - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source-stated The author says the evaluated task distribution may differ from real-world tasks. > the distribution of the types of tasks is probably moderately different in the real world compared to our task suite. - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source-stated A higher token budget barely changed a preliminary time horizon measurement for Opus 4.5 with Claude Code. > The preliminary time horizon measurement barely changed - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source-stated The author reports that GPT-5 with Codex sometimes addressed a user who was not present in the automated evaluation. > At the end of many runs, GPT-5 with Codex writes a short report about its work, saying things like “If you want, I can also commit these changes to git.” - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source and contributor context METR · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Time horizon METR's measure of AI agents' ability: the length of task, timed by how long skilled people take, that an agent completes at a set success rate. - [Definition reference](https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/) ## METR METR studies what AI systems can do and how reliably they complete tasks, including work with human developers. - [Explore this context](/entities/metr) - [Reference](https://metr.org/) ## Claude Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding. - [Explore this context](/entities/claude) - [Reference](https://claude.com/product/overview) - [Reference](https://www.anthropic.com/news) - [Reference](https://www.anthropic.com/research/team/economics) ## Editorial provenance Version 1. Publication ID: 988c3518-31d7-7d4d-4b60-8886b9f59aae. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-08T05:18:44.905Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model. Observation: 84bd2ce11f4bb46ce8b4ccc2ea2edc17d86776fc1c4f781ab8781eed0f7b5117. Source text hash: a8caa3bbef2a9e5ae8f6d8619ac81aa21c5045b709a35ae3fab3adfaf3a7afa5. ## Revision and source-check history ### Version 1 — 2026-11-08T05:23:23.785Z METR’s note, published February 13, 2026, reports that Claude Code and Codex were not obviously better than METR’s default scaffolds when measuring Opus 4.5 and GPT-5. The author concludes that these specialized setups “doesn’t make a huge difference” to the models’ measured time horizon, and thinks the difference probably is not huge for other recent models either. The author points to a key difference in how the setups are used: Claude Code and Codex are mostly used interactively, with a human checking the agent’s work, while METR’s tasks are autonomous. The task suite may also differ from real-world work. This leaves open whether the comparison would look different with human check-ins or a different task mix. The note’s examples include GPT-5 with Codex addressing a nonexistent user, and Opus 4.5 with Claude Code sticking with a poor plan or wasting limited attempts without using results. These examples describe observed runs, not how often such failures occur elsewhere. The author notes possible effects from token budgets, rough wrappers and technical issues; a higher token budget barely changed a preliminary measurement. The note’s conclusion is limited to the tested models and task setting. ## Original source METR · Article Original source date: Feb 13, 2026 (2026-02-13T08:00:00.000Z). Date basis: Date supplied by the publisher’s feed. Added to Escape Meridian: 2026-11-08T05:23:23.785Z. First observed: 2026-10-18T03:09:19.726Z. Retrieved: 2026-10-23T21:09:37.865Z. - [Measuring Time Horizon using Claude Code and Codex](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Limits Text as retrieved on October 23, 2026; originally published February 13, 2026. The comparison concerns Opus 4.5 and GPT-5 on METR’s task suite; the author says the real-world distribution of task types may differ. The author notes possible effects from token budgets, rough wrappers and minor technical issues. The note describes the findings as unpolished research updates that have gone through less review than METR’s research posts. The qualitative failure examples do not establish how often those behaviors occur outside the reported runs. ## Open question How would the comparison change if Claude Code and Codex were evaluated in the interactive settings where a human checks the agent’s work? ## Source-stated METR’s note reports that neither specialized scaffold was obviously better than METR’s default scaffolds for the tested models. > It looks like Claude Code and Codex aren’t obviously better than the default scaffolds we use for our agents. - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source-stated The author concludes that using Codex and Claude Code does not make a large difference to the measured time horizon of GPT-5 and Opus 4.5. > for GPT-5 and Opus 4.5, using Codex and Claude Code doesn’t make a huge difference for the time horizon of the model. - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source-stated The author says neither model’s difference was statistically significant in the comparison. > Neither the GPT-5 time horizon difference, nor the Opus 4.5 time horizon difference, are statistically significant: - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source-stated The author identifies the difference between interactive use and METR’s autonomous tasks as one reason the specialized scaffolds may not outperform the defaults in this evaluation. > Those scaffolds are mostly used in an interactive setting, where a human user often checks in on the agent’s work. In contrast, agents perform our time horizon tasks autonomously, without any human input. - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source-stated The author says the evaluated task distribution may differ from real-world tasks. > the distribution of the types of tasks is probably moderately different in the real world compared to our task suite. - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source-stated A higher token budget barely changed a preliminary time horizon measurement for Opus 4.5 with Claude Code. > The preliminary time horizon measurement barely changed - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source-stated The author reports that GPT-5 with Codex sometimes addressed a user who was not present in the automated evaluation. > At the end of many runs, GPT-5 with Codex writes a short report about its work, saying things like “If you want, I can also commit these changes to git.” - [Supporting source](https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/) ## Source and contributor context METR · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Time horizon METR's measure of AI agents' ability: the length of task, timed by how long skilled people take, that an agent completes at a set success rate. - [Definition reference](https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/) ## METR METR studies what AI systems can do and how reliably they complete tasks, including work with human developers. - [Explore this context](/entities/metr) - [Reference](https://metr.org/) ## Claude Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding. - [Explore this context](/entities/claude) - [Reference](https://claude.com/product/overview) - [Reference](https://www.anthropic.com/news) - [Reference](https://www.anthropic.com/research/team/economics) ## Editorial provenance Version 1. Publication ID: 988c3518-31d7-7d4d-4b60-8886b9f59aae. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-08T05:18:44.905Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model. Observation: 84bd2ce11f4bb46ce8b4ccc2ea2edc17d86776fc1c4f781ab8781eed0f7b5117. Source text hash: a8caa3bbef2a9e5ae8f6d8619ac81aa21c5045b709a35ae3fab3adfaf3a7afa5. ### uncertain — 2026-11-08T05:18:44.905Z — anthropic/claude-opus-5.5 Publication: 988c3518-31d7-7d4d-4b60-8886b9f59aae attribution — pass: The reading attributes the findings to METR's note and its author and keeps the conclusions as the author's. Its limitations disclose that notes are unpolished and less reviewed. The title's 'METR reports' is acceptable for a note on METR's site, though the source adds that notes do not necessarily reflect METR's views as a whole. > They do not necessarily reflect the views of METR as a whole. summary — uncertain: The central term 'scaffold' is never explained, and the definitions list does not cover it. Readers are not told that a scaffold is the agent wrapper or loop around a model that lets it take actions. They are also not told that Claude Code and Codex are Anthropic's and OpenAI's coding agent tools, or that ReAct and Triframe are METR's general-purpose ones. Without this, a non-specialist cannot readily grasp what was compared. Smallest repair: one plain-language sentence defining scaffold, or adding a term-library tooltip for it. Minor: the quoted 'doesn't' is grammatically mismatched with 'these specialized setups'. Otherwise the summary and question fairly reflect the source. > ReAct is a very simple scaffold where an agent takes an action, sees the results of the action, and repeats. date — pass: The reading's February 13, 2026 date matches the UTC day of evidence.originalPublished and the URL path. The source is a publisher page, not an archive copy. The reading uses no hindsight framing and does not present any forecast as an outcome. support — pass: Each claim matches the source: - no obvious advantage and no statistically significant difference - the interactive-versus-autonomous explanation and task-distribution caveat - the 32M token rerun that barely changed a preliminary measurement - the qualitative failure examples The reading adds no unsupported benefits or costs, and it notes that the examples do not establish frequency. > I decided to run another time horizon measurement with a four times higher token budget of 32 million tokens. The preliminary time horizon measurement barely changed media — pass: No media was selected, and no verified media candidates were supplied, so null is appropriate. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/a57da4457a978fefa5c65ae86bc65077fd972fe83b4bfc4e3e3ac2a0fafa421f/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/metr-reports-claude-code-and-codex-make-little-difference-in-a57da445.json)