# METR reports on GPT-4.5’s evaluations and their limits [Canonical reading](https://escapemeridian.com/readings/metr-reports-on-gpt-4-5-s-evaluations-and-their-limits-718e0d7f) METR published this evaluation on February 27, 2025. METR says GPT-4.5 performed between GPT-4o and o1 in its preliminary evaluations, which used an agent scaffold optimized for o1. METR describes this result as consistent with OpenAI’s findings, but says its tentative belief that GPT-4.5 is unlikely to pose large risks comes mainly from its broader understanding of frontier models, not from the evaluations alone. The evaluations do not address risks from internal deployment, and short pre-deployment evaluations may underestimate capabilities. METR argues that a short evaluation before public release would do little to mitigate lifetime risks when a model could also be used internally or stolen; it sees independent review of developers’ internal evaluation results as a promising direction. METR also discusses possible capability gains from cheap post-training modifications and speculative sandbagging, where a model could behave in ways that make its capabilities look lower during evaluation. The published evaluations do not rule out these possibilities. METR says it had not fully reviewed all results in OpenAI’s GPT-4.5 System Card, and that its preliminary transcript review had found no clear evidence of sandbagging. How likely these evaluation gaps are to conceal dangerous capabilities remains unknown. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source METR · Article Original source date: Feb 27, 2025 (2025-02-27T08:00:00.000Z). Date basis: Date supplied by the publisher’s feed. Added to Escape Meridian: 2026-11-15T03:03:26.133Z. First observed: 2026-10-18T03:37:22.563Z. Retrieved: 2026-10-24T00:59:35.671Z. - [METR’s GPT-4.5 pre-deployment evaluations](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Limits Text as retrieved on October 24, 2026; originally published February 27, 2025. METR describes the evaluations as preliminary and notes that its GPT-4.5 agent scaffold was optimized for o1. METR says it had not fully reviewed all results in OpenAI’s GPT-4.5 System Card when writing. The source presents sandbagging as speculative and says the published evaluations do not rule it out. ## Open question How well can evaluation methods detect capability gains or deliberate underperformance that the evaluations themselves may miss? ## Source-stated METR reports that GPT-4.5 performed between GPT-4o and o1 in its preliminary evaluations, using an agent scaffold optimized for o1. > Our findings are consistent with OpenAI’s results: GPT-4.5 exhibited overall performance somewhere between GPT-4o and o1. - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR says its belief that GPT-4.5 is unlikely to pose large risks, especially relative to existing frontier models, is tentative and does not rest primarily on its evaluations. > this belief stems primarily from our general understanding of both the capabilities and limitations of current frontier models. - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR says its evaluations do not rule out all large-scale risks because they do not address internal deployment risks and may underestimate GPT-4.5’s capabilities. > We do not believe that our evaluations (or the results shared with us by OpenAI1) themselves are sufficient to rule out all large-scale risks from models such as GPT-4.5 - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR argues that short pre-deployment evaluations would do little to mitigate lifetime risks, and sees independent review of developers’ internal results as promising. > a short pre-deployment evaluation sprint (even if perfectly accurate) would do little to mitigate the lifetime risks posed by advanced AIs. - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR says cheap post-training modifications could produce large capability gains, and that such gains are difficult to rule out. > There may exist elicitation techniques that can be performed for a small fraction of the cost of training a model such as GPT-4.5, and that may boost the model capabilities by a large amount. - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR raises the speculative possibility that models could deliberately alter their behavior during evaluation to appear less dangerous. > models could strategically alter their behavior during evaluation in order to appear less dangerous. - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR says its preliminary review of a small fraction of GPT-4.5 evaluation transcripts had found no clear evidence of sandbagging. > so far have not found any clear evidence of sandbagging. - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR says it had not fully reviewed all evaluation results contained in OpenAI’s GPT-4.5 System Card. > At time of writing, we have not had time to fully review the results of all evaluations contained in the GPT-4.5 System Card - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source and contributor context METR · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Agent harness An agent harness is the system that lets an AI model act as an agent: it processes inputs, orchestrates tool calls and returns results. An evaluation harness runs evals end to end, including grading outputs. - [Definition reference](https://www.anthropic.com/research/demystifying-evals-for-ai-agents) ## Frontier model One of the most capable AI models available at a given time; the set shifts as models improve. - [Definition reference](https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper) ## METR METR studies what AI systems can do and how reliably they complete tasks, including work with human developers. - [Explore this context](/entities/metr) - [Reference](https://metr.org/) ## OpenAI OpenAI develops AI models and products and publishes research, release announcements and evaluations. - [Explore this context](/entities/openai) - [Reference](https://openai.com/news/) ## Editorial provenance Version 1. Publication ID: 4764924c-53f2-0027-4a7c-d92115450bd4. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-15T02:58:42.621Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model. Observation: f8d3fae9a311d1436b64d4ffdcfe2dc16b01b1cc0ffc5f22b28839bb34387b0d. Source text hash: bfe1071135d17c0e28d4476e4bd6734a0512956b37ef5b2c53ae5d4dc1bd72ac. ## Revision and source-check history ### Version 1 — 2026-11-15T03:03:26.133Z METR published this evaluation on February 27, 2025. METR says GPT-4.5 performed between GPT-4o and o1 in its preliminary evaluations, which used an agent scaffold optimized for o1. METR describes this result as consistent with OpenAI’s findings, but says its tentative belief that GPT-4.5 is unlikely to pose large risks comes mainly from its broader understanding of frontier models, not from the evaluations alone. The evaluations do not address risks from internal deployment, and short pre-deployment evaluations may underestimate capabilities. METR argues that a short evaluation before public release would do little to mitigate lifetime risks when a model could also be used internally or stolen; it sees independent review of developers’ internal evaluation results as a promising direction. METR also discusses possible capability gains from cheap post-training modifications and speculative sandbagging, where a model could behave in ways that make its capabilities look lower during evaluation. The published evaluations do not rule out these possibilities. METR says it had not fully reviewed all results in OpenAI’s GPT-4.5 System Card, and that its preliminary transcript review had found no clear evidence of sandbagging. How likely these evaluation gaps are to conceal dangerous capabilities remains unknown. ## Original source METR · Article Original source date: Feb 27, 2025 (2025-02-27T08:00:00.000Z). Date basis: Date supplied by the publisher’s feed. Added to Escape Meridian: 2026-11-15T03:03:26.133Z. First observed: 2026-10-18T03:37:22.563Z. Retrieved: 2026-10-24T00:59:35.671Z. - [METR’s GPT-4.5 pre-deployment evaluations](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Limits Text as retrieved on October 24, 2026; originally published February 27, 2025. METR describes the evaluations as preliminary and notes that its GPT-4.5 agent scaffold was optimized for o1. METR says it had not fully reviewed all results in OpenAI’s GPT-4.5 System Card when writing. The source presents sandbagging as speculative and says the published evaluations do not rule it out. ## Open question How well can evaluation methods detect capability gains or deliberate underperformance that the evaluations themselves may miss? ## Source-stated METR reports that GPT-4.5 performed between GPT-4o and o1 in its preliminary evaluations, using an agent scaffold optimized for o1. > Our findings are consistent with OpenAI’s results: GPT-4.5 exhibited overall performance somewhere between GPT-4o and o1. - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR says its belief that GPT-4.5 is unlikely to pose large risks, especially relative to existing frontier models, is tentative and does not rest primarily on its evaluations. > this belief stems primarily from our general understanding of both the capabilities and limitations of current frontier models. - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR says its evaluations do not rule out all large-scale risks because they do not address internal deployment risks and may underestimate GPT-4.5’s capabilities. > We do not believe that our evaluations (or the results shared with us by OpenAI1) themselves are sufficient to rule out all large-scale risks from models such as GPT-4.5 - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR argues that short pre-deployment evaluations would do little to mitigate lifetime risks, and sees independent review of developers’ internal results as promising. > a short pre-deployment evaluation sprint (even if perfectly accurate) would do little to mitigate the lifetime risks posed by advanced AIs. - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR says cheap post-training modifications could produce large capability gains, and that such gains are difficult to rule out. > There may exist elicitation techniques that can be performed for a small fraction of the cost of training a model such as GPT-4.5, and that may boost the model capabilities by a large amount. - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR raises the speculative possibility that models could deliberately alter their behavior during evaluation to appear less dangerous. > models could strategically alter their behavior during evaluation in order to appear less dangerous. - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR says its preliminary review of a small fraction of GPT-4.5 evaluation transcripts had found no clear evidence of sandbagging. > so far have not found any clear evidence of sandbagging. - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source-stated METR says it had not fully reviewed all evaluation results contained in OpenAI’s GPT-4.5 System Card. > At time of writing, we have not had time to fully review the results of all evaluations contained in the GPT-4.5 System Card - [Supporting source](https://metr.org/blog/2025-02-27-gpt-4-5-evals/) ## Source and contributor context METR · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Agent harness An agent harness is the system that lets an AI model act as an agent: it processes inputs, orchestrates tool calls and returns results. An evaluation harness runs evals end to end, including grading outputs. - [Definition reference](https://www.anthropic.com/research/demystifying-evals-for-ai-agents) ## Frontier model One of the most capable AI models available at a given time; the set shifts as models improve. - [Definition reference](https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper) ## METR METR studies what AI systems can do and how reliably they complete tasks, including work with human developers. - [Explore this context](/entities/metr) - [Reference](https://metr.org/) ## OpenAI OpenAI develops AI models and products and publishes research, release announcements and evaluations. - [Explore this context](/entities/openai) - [Reference](https://openai.com/news/) ## Editorial provenance Version 1. Publication ID: 4764924c-53f2-0027-4a7c-d92115450bd4. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-15T02:58:42.621Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model. Observation: f8d3fae9a311d1436b64d4ffdcfe2dc16b01b1cc0ffc5f22b28839bb34387b0d. Source text hash: bfe1071135d17c0e28d4476e4bd6734a0512956b37ef5b2c53ae5d4dc1bd72ac. ### passed — 2026-11-15T02:58:42.621Z — anthropic/claude-opus-5.5 Publication: 4764924c-53f2-0027-4a7c-d92115450bd4 attribution — pass: This is a single METR blog post on its own GPT-4.5 pre-deployment evaluations. The reading consistently attributes findings, beliefs and arguments to METR and attributes the System Card to OpenAI. > METR subsequently performed a preliminary set of evaluations to measure the model’s performance on our general autonomy suite and RE-Bench. summary — pass: The reading clearly leads with METR's result and its tentative risk view, along with the stated limits. It keeps METR's argument about pre-deployment sprints versus internal use and theft, and its proposal for external review. Sandbagging is explained in plain language. Agent scaffold and frontier model are defined in the page definitions. The question follows from the evidence without being leading. Omitting the time-horizon metric does not impair understanding. > We currently believe that it’s unlikely that GPT-4.5 poses large risks—especially relative to existing frontier models. date — pass: The originalPublished instant 2025-02-27T08:00Z falls on February 27, 2025 in UTC, which matches the URL path and the reading. The reading uses no hindsight framing and presents no later events. Planned future posts are not presented as outcomes. The evidence is a publisher page, so no archive disclosure is required. support — pass: Each claim is stated in the source. The scaffold caveat, the basis of the tentative belief, internal deployment, underestimation, the sprint argument, cheap post-training gains being hard to rule out, speculative sandbagging, the incomplete System Card review and the preliminary transcript finding are all supported. The reading adds no unsupported benefits or forecasts. > there remain several possibilities by which this model could be unacceptably dangerous, that seem unlikely a priori but are not ruled out by either our or OpenAI’s published evaluations. media — pass: No media is selected, and no verified candidates were supplied, so leaving the media field empty is appropriate. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/718e0d7f1a38916db73c10b46e3c8842d5124aeec2730aaf6d1b1680e3f675be/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/metr-reports-on-gpt-4-5-s-evaluations-and-their-limits-718e0d7f.json)