{"id":"b739e1810b348b0c0cf07b28764617ebe65cbd434392acca4fbdd070543e7517","title":"METR reports on its review of OpenAI’s gpt-oss methodology","argument":"METR published this review on October 23, 2025. METR says it examined OpenAI’s methodology for testing whether malicious fine-tuning could bring gpt-oss-120b to “High” capability thresholds under specified threat models. The review covered Biological and Chemical and Cybersecurity, but not the model’s overall capabilities or the merits of releasing its weights. METR says ProtocolQA mattered to assessing whether models would be classified as High capability: OpenAI o3 had exceeded expert baselines in other biology evaluations, but not on ProtocolQA. METR reports that OpenAI incorporated nine of its recommendations and that it believes OpenAI at least partially addressed each high-urgency item. However, METR classifies clear criteria for High capability as only partially addressed. OpenAI said it had internal pre-registered thresholds, but did not share the thresholds, their rationale, or its preregistration procedure. METR hopes OpenAI will make its Preparedness risk thresholds available for external scrutiny. The review was conducted under a nondisclosure agreement requiring METR to share the post with OpenAI for review and approval. METR says the review did not cover risks involving actors with more resources than OpenAI’s threat model defined.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 29, 2026; originally published October 23, 2025. METR says the review did not assess the model’s overall capabilities or the merits of releasing its weights. METR says it was not asked to assess risks involving actors with more resources than those in OpenAI’s threat-model definition.","speaker":{"name":"METR","roleAtSource":"Publisher"},"source":{"title":"Summary of our gpt-oss methodology review","publisher":"METR","format":"Article","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","publishedAt":"2025-10-23T07:00:00.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"METR reviewed OpenAI’s methodology and results for experiments testing whether malicious fine-tuning could bring gpt-oss-120b to “High” capability thresholds under specified threat models.","excerpt":"METR reviewed the methodology behind the adversarial finetuning experiments that OpenAI conducted before releasing gpt-oss-120b. Our goal was to review the methodology and results to help them determine whether a malicious actor could fine-tune the model to exceed the dangerous-capability thresholds","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","relationship":"Source-stated"},{"text":"The review was intentionally limited in scope and did not assess the model’s overall capabilities or the merits of releasing its weights.","excerpt":"We did not attempt a holistic assessment of the model’s overall capabilities, nor did we evaluate the merits of releasing its weights.","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","relationship":"Source-stated"},{"text":"METR says OpenAI incorporated nine of its recommendations and that it believes OpenAI at least partially addressed each of the six high-urgency items.","excerpt":"indicating in the gpt-oss model card that they incorporated 9 of our 17.\nWe reviewed the published MFT paper and believe that OpenAI at least partially addressed each of our 6 high-urgency items.","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","relationship":"Source-stated"},{"text":"METR says OpenAI has not published clear criteria and justifications for classifying models as High capability; OpenAI later said it had internal pre-registered thresholds but did not share the thresholds, their rationale, or its preregistration procedure.","excerpt":"However, they did not share what these thresholds are, the reasoning behind them, or the preregistration procedure OpenAI followed.","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","relationship":"Source-stated"},{"text":"METR says its review was conducted under a nondisclosure agreement that required it to share the post with OpenAI for review and approval.","excerpt":"Our review was conducted under a nondisclosure agreement, which required us to share this post with OpenAI for review and approval.","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Will OpenAI make its Preparedness risk thresholds, their rationale, and its preregistration procedure available for external scrutiny?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"METR reports on its review of OpenAI’s gpt-oss methodology","summary":"METR published this review on October 23, 2025. METR says it examined OpenAI’s methodology for testing whether malicious fine-tuning could bring gpt-oss-120b to “High” capability thresholds under specified threat models. The review covered Biological and Chemical and Cybersecurity, but not the model’s overall capabilities or the merits of releasing its weights. METR says ProtocolQA mattered to assessing whether models would be classified as High capability: OpenAI o3 had exceeded expert baselines in other biology evaluations, but not on ProtocolQA. METR reports that OpenAI incorporated nine of its recommendations and that it believes OpenAI at least partially addressed each high-urgency item. However, METR classifies clear criteria for High capability as only partially addressed. OpenAI said it had internal pre-registered thresholds, but did not share the thresholds, their rationale, or its preregistration procedure. METR hopes OpenAI will make its Preparedness risk thresholds available for external scrutiny. The review was conducted under a nondisclosure agreement requiring METR to share the post with OpenAI for review and approval. METR says the review did not cover risks involving actors with more resources than OpenAI’s threat model defined.","question":"Will OpenAI make its Preparedness risk thresholds, their rationale, and its preregistration procedure available for external scrutiny?","claims":[{"text":"METR reviewed OpenAI’s methodology and results for experiments testing whether malicious fine-tuning could bring gpt-oss-120b to “High” capability thresholds under specified threat models.","excerpt":"METR reviewed the methodology behind the adversarial finetuning experiments that OpenAI conducted before releasing gpt-oss-120b. Our goal was to review the methodology and results to help them determine whether a malicious actor could fine-tune the model to exceed the dangerous-capability thresholds","support":"source-stated"},{"text":"The review was intentionally limited in scope and did not assess the model’s overall capabilities or the merits of releasing its weights.","excerpt":"We did not attempt a holistic assessment of the model’s overall capabilities, nor did we evaluate the merits of releasing its weights.","support":"source-stated"},{"text":"METR says OpenAI incorporated nine of its recommendations and that it believes OpenAI at least partially addressed each of the six high-urgency items.","excerpt":"indicating in the gpt-oss model card that they incorporated 9 of our 17.\nWe reviewed the published MFT paper and believe that OpenAI at least partially addressed each of our 6 high-urgency items.","support":"source-stated"},{"text":"METR says OpenAI has not published clear criteria and justifications for classifying models as High capability; OpenAI later said it had internal pre-registered thresholds but did not share the thresholds, their rationale, or its preregistration procedure.","excerpt":"However, they did not share what these thresholds are, the reasoning behind them, or the preregistration procedure OpenAI followed.","support":"source-stated"},{"text":"METR says its review was conducted under a nondisclosure agreement that required it to share the post with OpenAI for review and approval.","excerpt":"Our review was conducted under a nondisclosure agreement, which required us to share this post with OpenAI for review and approval.","support":"source-stated"}],"limitations":["Text as retrieved on October 29, 2026; originally published October 23, 2025.","METR says the review did not assess the model’s overall capabilities or the merits of releasing its weights.","METR says it was not asked to assess risks involving actors with more resources than those in OpenAI’s threat-model definition."],"mediaId":null,"correctionReason":null,"id":"b739e1810b348b0c0cf07b28764617ebe65cbd434392acca4fbdd070543e7517","publicationId":"231d3c36-bc38-fee1-b34d-6af4d5f64ac5","version":1,"sourceUrl":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","sourceTitle":"Summary of our gpt-oss methodology review","publisher":"METR","publishedAt":"2025-10-23T07:00:00.000Z","originalPublished":{"value":"2025-10-23T07:00:00.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T03:17:21.283Z","createdAt":"2026-11-11T19:23:26.769Z","historical":true,"entityIds":["metr"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"6e203c8fd29cec47f2b57ae2f56cd2e9e8bde680fe58ccd892e8a3c15fc609a7","rubricVersion":"unrecorded","provider":"anthropic","model":"claude-opus-5-5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-29T18:12:13.107Z"},"provenance":{"textHash":"2db77e5803ca6cf8732b27c63c706d81f74f1ad19fffba4c4f508f2892c3e98e","observationId":"dc4939ccda4a04a20f517d86ac9bb057e9946f99c3056adaf37f818e861cfe1f","retrievedAt":"2026-10-29T18:13:58.013Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"50d3058c-7899-4554-0eb7-3874455c2af6"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"5cd1f02a-1bad-e536-65aa-8a41110eaf14"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-11T19:18:41.715Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/metr-reports-on-its-review-of-openai-s-gpt-oss-methodology-b739e181","formats":{"html":"/readings/metr-reports-on-its-review-of-openai-s-gpt-oss-methodology-b739e181","markdown":"/readings/metr-reports-on-its-review-of-openai-s-gpt-oss-methodology-b739e181.md","json":"/readings/metr-reports-on-its-review-of-openai-s-gpt-oss-methodology-b739e181.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-fine-tuning","name":"Fine-tuning","explanation":"Further training of an existing model to adapt it to a particular task or use.","referenceUrl":"https://developers.google.com/machine-learning/glossary#fine-tuning"}],"entities":[{"id":"metr","name":"METR","kind":"organization","context":"METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.","referenceUrls":["https://metr.org/"],"path":"/entities/metr","json":"/entities/metr.json","markdown":"/entities/metr.md"},{"id":"openai","name":"OpenAI","kind":"organization","context":"OpenAI develops AI models and products and publishes research, release announcements and evaluations.","referenceUrls":["https://openai.com/news/"],"path":"/entities/openai","json":"/entities/openai.json","markdown":"/entities/openai.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"b739e1810b348b0c0cf07b28764617ebe65cbd434392acca4fbdd070543e7517","revision":"231d3c36-bc38-fee1-b34d-6af4d5f64ac5","revisionKind":"publication-id","title":"METR reports on its review of OpenAI’s gpt-oss methodology","url":"https://escapemeridian.com/readings/metr-reports-on-its-review-of-openai-s-gpt-oss-methodology-b739e181"},"source":{"type":"source","id":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","revision":"2db77e5803ca6cf8732b27c63c706d81f74f1ad19fffba4c4f508f2892c3e98e","revisionKind":"sha256-source-text","title":"Summary of our gpt-oss methodology review","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"b739e1810b348b0c0cf07b28764617ebe65cbd434392acca4fbdd070543e7517","title":"METR reports on its review of OpenAI’s gpt-oss methodology","argument":"METR published this review on October 23, 2025. METR says it examined OpenAI’s methodology for testing whether malicious fine-tuning could bring gpt-oss-120b to “High” capability thresholds under specified threat models. The review covered Biological and Chemical and Cybersecurity, but not the model’s overall capabilities or the merits of releasing its weights. METR says ProtocolQA mattered to assessing whether models would be classified as High capability: OpenAI o3 had exceeded expert baselines in other biology evaluations, but not on ProtocolQA. METR reports that OpenAI incorporated nine of its recommendations and that it believes OpenAI at least partially addressed each high-urgency item. However, METR classifies clear criteria for High capability as only partially addressed. OpenAI said it had internal pre-registered thresholds, but did not share the thresholds, their rationale, or its preregistration procedure. METR hopes OpenAI will make its Preparedness risk thresholds available for external scrutiny. The review was conducted under a nondisclosure agreement requiring METR to share the post with OpenAI for review and approval. METR says the review did not cover risks involving actors with more resources than OpenAI’s threat model defined.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 29, 2026; originally published October 23, 2025. METR says the review did not assess the model’s overall capabilities or the merits of releasing its weights. METR says it was not asked to assess risks involving actors with more resources than those in OpenAI’s threat-model definition.","speaker":{"name":"METR","roleAtSource":"Publisher"},"source":{"title":"Summary of our gpt-oss methodology review","publisher":"METR","format":"Article","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","publishedAt":"2025-10-23T07:00:00.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"METR reviewed OpenAI’s methodology and results for experiments testing whether malicious fine-tuning could bring gpt-oss-120b to “High” capability thresholds under specified threat models.","excerpt":"METR reviewed the methodology behind the adversarial finetuning experiments that OpenAI conducted before releasing gpt-oss-120b. Our goal was to review the methodology and results to help them determine whether a malicious actor could fine-tune the model to exceed the dangerous-capability thresholds","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","relationship":"Source-stated"},{"text":"The review was intentionally limited in scope and did not assess the model’s overall capabilities or the merits of releasing its weights.","excerpt":"We did not attempt a holistic assessment of the model’s overall capabilities, nor did we evaluate the merits of releasing its weights.","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","relationship":"Source-stated"},{"text":"METR says OpenAI incorporated nine of its recommendations and that it believes OpenAI at least partially addressed each of the six high-urgency items.","excerpt":"indicating in the gpt-oss model card that they incorporated 9 of our 17.\nWe reviewed the published MFT paper and believe that OpenAI at least partially addressed each of our 6 high-urgency items.","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","relationship":"Source-stated"},{"text":"METR says OpenAI has not published clear criteria and justifications for classifying models as High capability; OpenAI later said it had internal pre-registered thresholds but did not share the thresholds, their rationale, or its preregistration procedure.","excerpt":"However, they did not share what these thresholds are, the reasoning behind them, or the preregistration procedure OpenAI followed.","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","relationship":"Source-stated"},{"text":"METR says its review was conducted under a nondisclosure agreement that required it to share the post with OpenAI for review and approval.","excerpt":"Our review was conducted under a nondisclosure agreement, which required us to share this post with OpenAI for review and approval.","url":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Will OpenAI make its Preparedness risk thresholds, their rationale, and its preregistration procedure available for external scrutiny?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"METR reports on its review of OpenAI’s gpt-oss methodology","summary":"METR published this review on October 23, 2025. METR says it examined OpenAI’s methodology for testing whether malicious fine-tuning could bring gpt-oss-120b to “High” capability thresholds under specified threat models. The review covered Biological and Chemical and Cybersecurity, but not the model’s overall capabilities or the merits of releasing its weights. METR says ProtocolQA mattered to assessing whether models would be classified as High capability: OpenAI o3 had exceeded expert baselines in other biology evaluations, but not on ProtocolQA. METR reports that OpenAI incorporated nine of its recommendations and that it believes OpenAI at least partially addressed each high-urgency item. However, METR classifies clear criteria for High capability as only partially addressed. OpenAI said it had internal pre-registered thresholds, but did not share the thresholds, their rationale, or its preregistration procedure. METR hopes OpenAI will make its Preparedness risk thresholds available for external scrutiny. The review was conducted under a nondisclosure agreement requiring METR to share the post with OpenAI for review and approval. METR says the review did not cover risks involving actors with more resources than OpenAI’s threat model defined.","question":"Will OpenAI make its Preparedness risk thresholds, their rationale, and its preregistration procedure available for external scrutiny?","claims":[{"text":"METR reviewed OpenAI’s methodology and results for experiments testing whether malicious fine-tuning could bring gpt-oss-120b to “High” capability thresholds under specified threat models.","excerpt":"METR reviewed the methodology behind the adversarial finetuning experiments that OpenAI conducted before releasing gpt-oss-120b. Our goal was to review the methodology and results to help them determine whether a malicious actor could fine-tune the model to exceed the dangerous-capability thresholds","support":"source-stated"},{"text":"The review was intentionally limited in scope and did not assess the model’s overall capabilities or the merits of releasing its weights.","excerpt":"We did not attempt a holistic assessment of the model’s overall capabilities, nor did we evaluate the merits of releasing its weights.","support":"source-stated"},{"text":"METR says OpenAI incorporated nine of its recommendations and that it believes OpenAI at least partially addressed each of the six high-urgency items.","excerpt":"indicating in the gpt-oss model card that they incorporated 9 of our 17.\nWe reviewed the published MFT paper and believe that OpenAI at least partially addressed each of our 6 high-urgency items.","support":"source-stated"},{"text":"METR says OpenAI has not published clear criteria and justifications for classifying models as High capability; OpenAI later said it had internal pre-registered thresholds but did not share the thresholds, their rationale, or its preregistration procedure.","excerpt":"However, they did not share what these thresholds are, the reasoning behind them, or the preregistration procedure OpenAI followed.","support":"source-stated"},{"text":"METR says its review was conducted under a nondisclosure agreement that required it to share the post with OpenAI for review and approval.","excerpt":"Our review was conducted under a nondisclosure agreement, which required us to share this post with OpenAI for review and approval.","support":"source-stated"}],"limitations":["Text as retrieved on October 29, 2026; originally published October 23, 2025.","METR says the review did not assess the model’s overall capabilities or the merits of releasing its weights.","METR says it was not asked to assess risks involving actors with more resources than those in OpenAI’s threat-model definition."],"mediaId":null,"correctionReason":null,"id":"b739e1810b348b0c0cf07b28764617ebe65cbd434392acca4fbdd070543e7517","publicationId":"231d3c36-bc38-fee1-b34d-6af4d5f64ac5","version":1,"sourceUrl":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","sourceTitle":"Summary of our gpt-oss methodology review","publisher":"METR","publishedAt":"2025-10-23T07:00:00.000Z","originalPublished":{"value":"2025-10-23T07:00:00.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T03:17:21.283Z","createdAt":"2026-11-11T19:23:26.769Z","historical":true,"entityIds":["metr"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://metr.org/blog/2025-10-23-gpt-oss-methodology-review/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"6e203c8fd29cec47f2b57ae2f56cd2e9e8bde680fe58ccd892e8a3c15fc609a7","rubricVersion":"unrecorded","provider":"anthropic","model":"claude-opus-5-5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-29T18:12:13.107Z"},"provenance":{"textHash":"2db77e5803ca6cf8732b27c63c706d81f74f1ad19fffba4c4f508f2892c3e98e","observationId":"dc4939ccda4a04a20f517d86ac9bb057e9946f99c3056adaf37f818e861cfe1f","retrievedAt":"2026-10-29T18:13:58.013Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"50d3058c-7899-4554-0eb7-3874455c2af6"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"5cd1f02a-1bad-e536-65aa-8a41110eaf14"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-11T19:18:41.715Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"231d3c36-bc38-fee1-b34d-6af4d5f64ac5","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading consistently attributes the review, findings and hopes to METR, and OpenAI's statements to OpenAI as METR reports them. It is a single first-person METR post, not a listing.","excerpt":"METR reviewed the methodology behind the adversarial finetuning experiments that OpenAI conducted before releasing gpt-oss-120b."},{"dimension":"summary","status":"uncertain","explanation":"The reading is faithful but leaves central terms unexplained for a non-specialist. ProtocolQA carries a key passage yet is never described; the source says it measures error detection in biology protocols. The OpenAI Preparedness Framework is never named, so 'Preparedness risk thresholds' and 'High' capability levels lack context. Smallest repair: add METR's description of ProtocolQA, name the Preparedness Framework, and add term-library entries for ProtocolQA, Preparedness Framework and threat model. Also consider noting the 17-recommendation total, and that METR removed one reference at OpenAI's request while saying its conclusions did not change.","excerpt":"OpenAI stated that ProtocolQA measured error detection capabilities their other benchmarks could not."},{"dimension":"date","status":"pass","explanation":"The originalPublished instant 2025-10-23T07:00Z falls on October 23, 2025 in UTC, which matches the reading and the URL path. There is no hindsight framing. The source kind is publisher-page, so no archive disclosure is needed. The retrieval date is labeled as such.","excerpt":null},{"dimension":"support","status":"pass","explanation":"Each claim is stated in the source. These include the scope limits, nine incorporated recommendations, at least partial handling of all six high-urgency items, and the undisclosed pre-registered thresholds. They also include the NDA and approval requirement and the threat-model resource limit. The question restates METR's own stated hope without implying an outcome.","excerpt":"In the future, we hope that they will make their Preparedness risk thresholds available for external scrutiny."},{"dimension":"media","status":"pass","explanation":"No media was selected, and no verified candidates were supplied.","excerpt":null}],"createdAt":"2026-11-11T19:18:41.715Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}