· Feb 27, 2025
METR reports on GPT-4.5’s evaluations and their limits
Markdown · JSON · Original source
published this evaluation on February 27, 2025. METR says GPT-4.5 performed between GPT-4o and o1 in its preliminary evaluations, which used an optimized for o1. METR describes this result as consistent with ’s findings, but says its tentative belief that GPT-4.5 is unlikely to pose large risks comes mainly from its broader understanding of , not from the evaluations alone. The evaluations do not address risks from internal deployment, and short pre-deployment evaluations may underestimate capabilities. METR argues that a short evaluation before public release would do little to mitigate lifetime risks when a model could also be used internally or stolen; it sees independent review of developers’ internal evaluation results as a promising direction. METR also discusses possible capability gains from cheap post-training modifications and speculative sandbagging, where a model could behave in ways that make its capabilities look lower during evaluation. The published evaluations do not rule out these possibilities. METR says it had not fully reviewed all results in OpenAI’s GPT-4.5 System Card, and that its preliminary transcript review had found no clear evidence of sandbagging. How likely these evaluation gaps are to conceal dangerous capabilities remains unknown.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
· Article
Source details
Original source date: Feb 27, 2025 (2025-02-27T08:00:00.000Z).
Date basis: Date supplied by the publisher’s feed.
Added to Escape Meridian: 2026-11-15T03:03:26.133Z. First observed: 2026-10-18T03:37:22.563Z. Retrieved: 2026-10-24T00:59:35.671Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
Text as retrieved on October 24, 2026; originally published February 27, 2025. describes the evaluations as preliminary and notes that its GPT-4.5 was optimized for o1. METR says it had not fully reviewed all results in ’s GPT-4.5 System Card when writing. The source presents sandbagging as speculative and says the published evaluations do not rule it out.
Open question
How well can evaluation methods detect capability gains or deliberate underperformance that the evaluations themselves may miss?
Source-stated
METR reports that GPT-4.5 performed between GPT-4o and o1 in its preliminary evaluations, using an agent scaffold optimized for o1.
Our findings are consistent with OpenAI’s results: GPT-4.5 exhibited overall performance somewhere between GPT-4o and o1.
Source-stated
METR says its belief that GPT-4.5 is unlikely to pose large risks, especially relative to existing frontier models, is tentative and does not rest primarily on its evaluations.
this belief stems primarily from our general understanding of both the capabilities and limitations of current frontier models.
Source-stated
METR says its evaluations do not rule out all large-scale risks because they do not address internal deployment risks and may underestimate GPT-4.5’s capabilities.
We do not believe that our evaluations (or the results shared with us by OpenAI1) themselves are sufficient to rule out all large-scale risks from models such as GPT-4.5
Source-stated
METR argues that short pre-deployment evaluations would do little to mitigate lifetime risks, and sees independent review of developers’ internal results as promising.
a short pre-deployment evaluation sprint (even if perfectly accurate) would do little to mitigate the lifetime risks posed by advanced AIs.
Source-stated
METR says cheap post-training modifications could produce large capability gains, and that such gains are difficult to rule out.
There may exist elicitation techniques that can be performed for a small fraction of the cost of training a model such as GPT-4.5, and that may boost the model capabilities by a large amount.
Source-stated
METR raises the speculative possibility that models could deliberately alter their behavior during evaluation to appear less dangerous.
models could strategically alter their behavior during evaluation in order to appear less dangerous.
Source-stated
METR says its preliminary review of a small fraction of GPT-4.5 evaluation transcripts had found no clear evidence of sandbagging.
so far have not found any clear evidence of sandbagging.
Source-stated
METR says it had not fully reviewed all evaluation results contained in OpenAI’s GPT-4.5 System Card.
At time of writing, we have not had time to fully review the results of all evaluations contained in the GPT-4.5 System Card
Source and contributor context
METR · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Agent harness
An agent harness is the system that lets an AI model act as an agent: it processes inputs, orchestrates tool calls and returns results. An evaluation harness runs evals end to end, including grading outputs.
Frontier model
One of the most capable AI models available at a given time; the set shifts as models improve.
METR
METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
OpenAI
OpenAI develops AI models and products and publishes research, release announcements and evaluations.
Editorial provenance
Version 1. Publication ID: 4764924c-53f2-0027-4a7c-d92115450bd4.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-15T02:58:42.621Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model.
Observation: f8d3fae9a311d1436b64d4ffdcfe2dc16b01b1cc0ffc5f22b28839bb34387b0d. Source text hash: bfe1071135d17c0e28d4476e4bd6734a0512956b37ef5b2c53ae5d4dc1bd72ac.
Revision and source-check history
Version 1 · 2026-11-15T03:03:26.133Z
METR reports on GPT-4.5’s evaluations and their limits
published this evaluation on February 27, 2025. METR says GPT-4.5 performed between GPT-4o and o1 in its preliminary evaluations, which used an optimized for o1. METR describes this result as consistent with ’s findings, but says its tentative belief that GPT-4.5 is unlikely to pose large risks comes mainly from its broader understanding of , not from the evaluations alone. The evaluations do not address risks from internal deployment, and short pre-deployment evaluations may underestimate capabilities. METR argues that a short evaluation before public release would do little to mitigate lifetime risks when a model could also be used internally or stolen; it sees independent review of developers’ internal evaluation results as a promising direction. METR also discusses possible capability gains from cheap post-training modifications and speculative sandbagging, where a model could behave in ways that make its capabilities look lower during evaluation. The published evaluations do not rule out these possibilities. METR says it had not fully reviewed all results in OpenAI’s GPT-4.5 System Card, and that its preliminary transcript review had found no clear evidence of sandbagging. How likely these evaluation gaps are to conceal dangerous capabilities remains unknown.
Original source
· Article
Source details
Original source date: Feb 27, 2025 (2025-02-27T08:00:00.000Z).
Date basis: Date supplied by the publisher’s feed.
Added to Escape Meridian: 2026-11-15T03:03:26.133Z. First observed: 2026-10-18T03:37:22.563Z. Retrieved: 2026-10-24T00:59:35.671Z.
Limits
Text as retrieved on October 24, 2026; originally published February 27, 2025. describes the evaluations as preliminary and notes that its GPT-4.5 was optimized for o1. METR says it had not fully reviewed all results in ’s GPT-4.5 System Card when writing. The source presents sandbagging as speculative and says the published evaluations do not rule it out.
Open question
How well can evaluation methods detect capability gains or deliberate underperformance that the evaluations themselves may miss?
Source-stated
METR reports that GPT-4.5 performed between GPT-4o and o1 in its preliminary evaluations, using an agent scaffold optimized for o1.
Our findings are consistent with OpenAI’s results: GPT-4.5 exhibited overall performance somewhere between GPT-4o and o1.
Source-stated
METR says its belief that GPT-4.5 is unlikely to pose large risks, especially relative to existing frontier models, is tentative and does not rest primarily on its evaluations.
this belief stems primarily from our general understanding of both the capabilities and limitations of current frontier models.
Source-stated
METR says its evaluations do not rule out all large-scale risks because they do not address internal deployment risks and may underestimate GPT-4.5’s capabilities.
We do not believe that our evaluations (or the results shared with us by OpenAI1) themselves are sufficient to rule out all large-scale risks from models such as GPT-4.5
Source-stated
METR argues that short pre-deployment evaluations would do little to mitigate lifetime risks, and sees independent review of developers’ internal results as promising.
a short pre-deployment evaluation sprint (even if perfectly accurate) would do little to mitigate the lifetime risks posed by advanced AIs.
Source-stated
METR says cheap post-training modifications could produce large capability gains, and that such gains are difficult to rule out.
There may exist elicitation techniques that can be performed for a small fraction of the cost of training a model such as GPT-4.5, and that may boost the model capabilities by a large amount.
Source-stated
METR raises the speculative possibility that models could deliberately alter their behavior during evaluation to appear less dangerous.
models could strategically alter their behavior during evaluation in order to appear less dangerous.
Source-stated
METR says its preliminary review of a small fraction of GPT-4.5 evaluation transcripts had found no clear evidence of sandbagging.
so far have not found any clear evidence of sandbagging.
Source-stated
METR says it had not fully reviewed all evaluation results contained in OpenAI’s GPT-4.5 System Card.
At time of writing, we have not had time to fully review the results of all evaluations contained in the GPT-4.5 System Card
Source and contributor context
METR · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Agent harness
An agent harness is the system that lets an AI model act as an agent: it processes inputs, orchestrates tool calls and returns results. An evaluation harness runs evals end to end, including grading outputs.
Frontier model
One of the most capable AI models available at a given time; the set shifts as models improve.
METR
METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
OpenAI
OpenAI develops AI models and products and publishes research, release announcements and evaluations.
Editorial provenance
Version 1. Publication ID: 4764924c-53f2-0027-4a7c-d92115450bd4.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-15T02:58:42.621Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model.
Observation: f8d3fae9a311d1436b64d4ffdcfe2dc16b01b1cc0ffc5f22b28839bb34387b0d. Source text hash: bfe1071135d17c0e28d4476e4bd6734a0512956b37ef5b2c53ae5d4dc1bd72ac.
passed · 2026-11-15T02:58:42.621Z · anthropic/claude-opus-5.5
Publication: 4764924c-53f2-0027-4a7c-d92115450bd4
attribution — pass: This is a single METR blog post on its own GPT-4.5 pre-deployment evaluations. The reading consistently attributes findings, beliefs and arguments to METR and attributes the System Card to OpenAI.
METR subsequently performed a preliminary set of evaluations to measure the model’s performance on our general autonomy suite and RE-Bench.
summary — pass: The reading clearly leads with METR's result and its tentative risk view, along with the stated limits. It keeps METR's argument about pre-deployment sprints versus internal use and theft, and its proposal for external review. Sandbagging is explained in plain language. Agent scaffold and frontier model are defined in the page definitions. The question follows from the evidence without being leading. Omitting the time-horizon metric does not impair understanding.
We currently believe that it’s unlikely that GPT-4.5 poses large risks—especially relative to existing frontier models.
date — pass: The originalPublished instant 2025-02-27T08:00Z falls on February 27, 2025 in UTC, which matches the URL path and the reading. The reading uses no hindsight framing and presents no later events. Planned future posts are not presented as outcomes. The evidence is a publisher page, so no archive disclosure is required.
support — pass: Each claim is stated in the source. The scaffold caveat, the basis of the tentative belief, internal deployment, underestimation, the sprint argument, cheap post-training gains being hard to rule out, speculative sandbagging, the incomplete System Card review and the preliminary transcript finding are all supported. The reading adds no unsupported benefits or forecasts.
there remain several possibilities by which this model could be unacceptably dangerous, that seem unlikely a priori but are not ruled out by either our or OpenAI’s published evaluations.
media — pass: No media is selected, and no verified candidates were supplied, so leaving the media field empty is appropriate.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRAn agent harness is the system that lets an AI model act as an agent: it processes inputs, orchestrates tool calls and returns results. An evaluation harness runs evals end to end, including grading outputs.
Definition referenceOpenAI develops AI models and products and publishes research, release announcements and evaluations.
Explore OpenAIOne of the most capable AI models available at a given time; the set shifts as models improve.
Definition referenceMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRAn agent harness is the system that lets an AI model act as an agent: it processes inputs, orchestrates tool calls and returns results. An evaluation harness runs evals end to end, including grading outputs.
Definition referenceOpenAI develops AI models and products and publishes research, release announcements and evaluations.
Explore OpenAIMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRAn agent harness is the system that lets an AI model act as an agent: it processes inputs, orchestrates tool calls and returns results. An evaluation harness runs evals end to end, including grading outputs.
Definition referenceOpenAI develops AI models and products and publishes research, release announcements and evaluations.
Explore OpenAIOne of the most capable AI models available at a given time; the set shifts as models improve.
Definition referenceMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRMETR studies what AI systems can do and how reliably they complete tasks, including work with human developers.
Explore METRAn agent harness is the system that lets an AI model act as an agent: it processes inputs, orchestrates tool calls and returns results. An evaluation harness runs evals end to end, including grading outputs.
Definition referenceOpenAI develops AI models and products and publishes research, release announcements and evaluations.
Explore OpenAI