{"id":"dbbe08f61c97553b6d2458312ec4185da814bf03dc0baba9cfa147ecce1eb220","title":"Epoch AI describes comparing open- and closed-weight models on cybersecurity benchmarks","argument":"Epoch AI’s page was published on October 16, 2026. It describes comparing open- and closed-weight models with the Cyber ECI, a cybersecurity-specific version of the Epoch Capabilities Index (ECI). Epoch AI says it keeps benchmark parameters from a joint fit using general and cyber benchmarks, then estimates each model’s Cyber ECI from its cybersecurity results alone; scores use the same scale as the general ECI. The page links to data, analysis, assumptions and limitations, and methodology documentation, but the retrieved text does not include the results or explain the size of the gap named in the page title. The retrieved text therefore leaves unknown what Cyber ECI scores and analysis support the stated comparison between open- and closed-weight models. The comparison describes a benchmark-based measure, not evidence here about models’ performance in real-world cybersecurity work.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 29, 2026; originally published October 16, 2026. The retrieved page text does not include the data, analysis, or methodology details, so it does not substantiate the size of the comparison named in the page title or show how the scores relate to real-world cybersecurity performance.","speaker":{"name":"Epoch AI","roleAtSource":"Publisher"},"source":{"title":"Open-weight models trail closed models in cyber capabilities by 6–7 months","publisher":"Epoch AI","format":"Article","url":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","publishedAt":"2026-10-16T00:00:00.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Epoch AI says it compares open- and closed-weight models using the Cyber ECI, a domain-specific version of the Epoch Capabilities Index based on cybersecurity benchmarks.","excerpt":"We compare open- and closed-weight models using the Cyber ECI, a domain-specific version of the Epoch Capabilities Index (ECI) based on cybersecurity benchmarks.","url":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","relationship":"Source-stated"},{"text":"Epoch AI says it estimates each model’s Cyber ECI from cyber results alone, while retaining benchmark parameters from a joint fit on general and cyber benchmarks.","excerpt":"We keep the benchmark parameters from a joint fit of the ECI on general and cyber benchmarks, and estimate each model’s Cyber ECI from its cyber results alone.","url":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","relationship":"Source-stated"},{"text":"Epoch AI says Cyber ECI scores use the same scale as the general ECI.","excerpt":"Scores are on the same scale as the general ECI.","url":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What Cyber ECI scores and analysis support the stated comparison between open- and closed-weight models?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Epoch AI describes comparing open- and closed-weight models on cybersecurity benchmarks","summary":"Epoch AI’s page was published on October 16, 2026. It describes comparing open- and closed-weight models with the Cyber ECI, a cybersecurity-specific version of the Epoch Capabilities Index (ECI). Epoch AI says it keeps benchmark parameters from a joint fit using general and cyber benchmarks, then estimates each model’s Cyber ECI from its cybersecurity results alone; scores use the same scale as the general ECI. The page links to data, analysis, assumptions and limitations, and methodology documentation, but the retrieved text does not include the results or explain the size of the gap named in the page title. The retrieved text therefore leaves unknown what Cyber ECI scores and analysis support the stated comparison between open- and closed-weight models. The comparison describes a benchmark-based measure, not evidence here about models’ performance in real-world cybersecurity work.","question":"What Cyber ECI scores and analysis support the stated comparison between open- and closed-weight models?","claims":[{"text":"Epoch AI says it compares open- and closed-weight models using the Cyber ECI, a domain-specific version of the Epoch Capabilities Index based on cybersecurity benchmarks.","excerpt":"We compare open- and closed-weight models using the Cyber ECI, a domain-specific version of the Epoch Capabilities Index (ECI) based on cybersecurity benchmarks.","support":"source-stated"},{"text":"Epoch AI says it estimates each model’s Cyber ECI from cyber results alone, while retaining benchmark parameters from a joint fit on general and cyber benchmarks.","excerpt":"We keep the benchmark parameters from a joint fit of the ECI on general and cyber benchmarks, and estimate each model’s Cyber ECI from its cyber results alone.","support":"source-stated"},{"text":"Epoch AI says Cyber ECI scores use the same scale as the general ECI.","excerpt":"Scores are on the same scale as the general ECI.","support":"source-stated"}],"limitations":["Text as retrieved on October 29, 2026; originally published October 16, 2026.","The retrieved page text does not include the data, analysis, or methodology details, so it does not substantiate the size of the comparison named in the page title or show how the scores relate to real-world cybersecurity performance."],"mediaId":null,"correctionReason":null,"id":"dbbe08f61c97553b6d2458312ec4185da814bf03dc0baba9cfa147ecce1eb220","publicationId":"3f3b1987-9724-ce0b-96b0-3866e5b88d9b","version":1,"sourceUrl":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","sourceTitle":"Open-weight models trail closed models in cyber capabilities by 6–7 months","publisher":"Epoch AI","publishedAt":"2026-10-16T00:00:00.000Z","originalPublished":{"value":"2026-10-16T00:00:00.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T02:34:56.924Z","createdAt":"2026-11-04T21:48:27.563Z","historical":true,"entityIds":["epoch-ai"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"aa682b9ee7d381b84ff6433c1e81f40a63a6fc9a757ccde978f926a48cc398ba","rubricVersion":"unrecorded","provider":"anthropic","model":"claude-opus-5-5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-29T12:03:57.742Z"},"provenance":{"textHash":"a0c28864e34539d63f7a87f8966781dce04d30688729e2e91ec8dd2720ee7838","observationId":"caaf879387d498f2fd9e897b7e63f5ccc8fea88379216062f66ea8ddfff653ff","retrievedAt":"2026-10-29T12:05:44.669Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"47e63d67-8da6-561b-21c8-5f03bc84601b"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"d0e4c226-547d-0105-eeee-f2ef38f02579"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-04T21:43:50.961Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/epoch-ai-describes-comparing-open-and-closed-weight-models-on-dbbe08f6","formats":{"html":"/readings/epoch-ai-describes-comparing-open-and-closed-weight-models-on-dbbe08f6","markdown":"/readings/epoch-ai-describes-comparing-open-and-closed-weight-models-on-dbbe08f6.md","json":"/readings/epoch-ai-describes-comparing-open-and-closed-weight-models-on-dbbe08f6.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"},{"id":"term-eci","name":"Epoch Capabilities Index","explanation":"Epoch AI's index that combines scores from many AI benchmarks into a single general-capability scale, so models can be compared even over periods long enough for single benchmarks to saturate.","referenceUrl":"https://epoch.ai/data/eci-documentation"}],"entities":[{"id":"epoch-ai","name":"Epoch AI","kind":"organization","context":"Epoch AI studies trends in AI capability, training resources and the infrastructure behind them.","referenceUrls":["https://epoch.ai/","https://epoch.ai/data"],"path":"/entities/epoch-ai","json":"/entities/epoch-ai.json","markdown":"/entities/epoch-ai.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"dbbe08f61c97553b6d2458312ec4185da814bf03dc0baba9cfa147ecce1eb220","revision":"3f3b1987-9724-ce0b-96b0-3866e5b88d9b","revisionKind":"publication-id","title":"Epoch AI describes comparing open- and closed-weight models on cybersecurity benchmarks","url":"https://escapemeridian.com/readings/epoch-ai-describes-comparing-open-and-closed-weight-models-on-dbbe08f6"},"source":{"type":"source","id":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","revision":"a0c28864e34539d63f7a87f8966781dce04d30688729e2e91ec8dd2720ee7838","revisionKind":"sha256-source-text","title":"Open-weight models trail closed models in cyber capabilities by 6–7 months","url":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"dbbe08f61c97553b6d2458312ec4185da814bf03dc0baba9cfa147ecce1eb220","title":"Epoch AI describes comparing open- and closed-weight models on cybersecurity benchmarks","argument":"Epoch AI’s page was published on October 16, 2026. It describes comparing open- and closed-weight models with the Cyber ECI, a cybersecurity-specific version of the Epoch Capabilities Index (ECI). Epoch AI says it keeps benchmark parameters from a joint fit using general and cyber benchmarks, then estimates each model’s Cyber ECI from its cybersecurity results alone; scores use the same scale as the general ECI. The page links to data, analysis, assumptions and limitations, and methodology documentation, but the retrieved text does not include the results or explain the size of the gap named in the page title. The retrieved text therefore leaves unknown what Cyber ECI scores and analysis support the stated comparison between open- and closed-weight models. The comparison describes a benchmark-based measure, not evidence here about models’ performance in real-world cybersecurity work.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 29, 2026; originally published October 16, 2026. The retrieved page text does not include the data, analysis, or methodology details, so it does not substantiate the size of the comparison named in the page title or show how the scores relate to real-world cybersecurity performance.","speaker":{"name":"Epoch AI","roleAtSource":"Publisher"},"source":{"title":"Open-weight models trail closed models in cyber capabilities by 6–7 months","publisher":"Epoch AI","format":"Article","url":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","publishedAt":"2026-10-16T00:00:00.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Epoch AI says it compares open- and closed-weight models using the Cyber ECI, a domain-specific version of the Epoch Capabilities Index based on cybersecurity benchmarks.","excerpt":"We compare open- and closed-weight models using the Cyber ECI, a domain-specific version of the Epoch Capabilities Index (ECI) based on cybersecurity benchmarks.","url":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","relationship":"Source-stated"},{"text":"Epoch AI says it estimates each model’s Cyber ECI from cyber results alone, while retaining benchmark parameters from a joint fit on general and cyber benchmarks.","excerpt":"We keep the benchmark parameters from a joint fit of the ECI on general and cyber benchmarks, and estimate each model’s Cyber ECI from its cyber results alone.","url":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","relationship":"Source-stated"},{"text":"Epoch AI says Cyber ECI scores use the same scale as the general ECI.","excerpt":"Scores are on the same scale as the general ECI.","url":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What Cyber ECI scores and analysis support the stated comparison between open- and closed-weight models?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Epoch AI describes comparing open- and closed-weight models on cybersecurity benchmarks","summary":"Epoch AI’s page was published on October 16, 2026. It describes comparing open- and closed-weight models with the Cyber ECI, a cybersecurity-specific version of the Epoch Capabilities Index (ECI). Epoch AI says it keeps benchmark parameters from a joint fit using general and cyber benchmarks, then estimates each model’s Cyber ECI from its cybersecurity results alone; scores use the same scale as the general ECI. The page links to data, analysis, assumptions and limitations, and methodology documentation, but the retrieved text does not include the results or explain the size of the gap named in the page title. The retrieved text therefore leaves unknown what Cyber ECI scores and analysis support the stated comparison between open- and closed-weight models. The comparison describes a benchmark-based measure, not evidence here about models’ performance in real-world cybersecurity work.","question":"What Cyber ECI scores and analysis support the stated comparison between open- and closed-weight models?","claims":[{"text":"Epoch AI says it compares open- and closed-weight models using the Cyber ECI, a domain-specific version of the Epoch Capabilities Index based on cybersecurity benchmarks.","excerpt":"We compare open- and closed-weight models using the Cyber ECI, a domain-specific version of the Epoch Capabilities Index (ECI) based on cybersecurity benchmarks.","support":"source-stated"},{"text":"Epoch AI says it estimates each model’s Cyber ECI from cyber results alone, while retaining benchmark parameters from a joint fit on general and cyber benchmarks.","excerpt":"We keep the benchmark parameters from a joint fit of the ECI on general and cyber benchmarks, and estimate each model’s Cyber ECI from its cyber results alone.","support":"source-stated"},{"text":"Epoch AI says Cyber ECI scores use the same scale as the general ECI.","excerpt":"Scores are on the same scale as the general ECI.","support":"source-stated"}],"limitations":["Text as retrieved on October 29, 2026; originally published October 16, 2026.","The retrieved page text does not include the data, analysis, or methodology details, so it does not substantiate the size of the comparison named in the page title or show how the scores relate to real-world cybersecurity performance."],"mediaId":null,"correctionReason":null,"id":"dbbe08f61c97553b6d2458312ec4185da814bf03dc0baba9cfa147ecce1eb220","publicationId":"3f3b1987-9724-ce0b-96b0-3866e5b88d9b","version":1,"sourceUrl":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","sourceTitle":"Open-weight models trail closed models in cyber capabilities by 6–7 months","publisher":"Epoch AI","publishedAt":"2026-10-16T00:00:00.000Z","originalPublished":{"value":"2026-10-16T00:00:00.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T02:34:56.924Z","createdAt":"2026-11-04T21:48:27.563Z","historical":true,"entityIds":["epoch-ai"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://epoch.ai/data-insights/open-closed-cyber-eci-gap","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"aa682b9ee7d381b84ff6433c1e81f40a63a6fc9a757ccde978f926a48cc398ba","rubricVersion":"unrecorded","provider":"anthropic","model":"claude-opus-5-5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-29T12:03:57.742Z"},"provenance":{"textHash":"a0c28864e34539d63f7a87f8966781dce04d30688729e2e91ec8dd2720ee7838","observationId":"caaf879387d498f2fd9e897b7e63f5ccc8fea88379216062f66ea8ddfff653ff","retrievedAt":"2026-10-29T12:05:44.669Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"47e63d67-8da6-561b-21c8-5f03bc84601b"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"d0e4c226-547d-0105-eeee-f2ef38f02579"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-04T21:43:50.961Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"3f3b1987-9724-ce0b-96b0-3866e5b88d9b","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The page is a single Epoch AI data insight describing its own Cyber ECI comparison, not a listing of other pieces. The reading attributes the methodology statements to Epoch AI, which matches the publisher and the first-person text.","excerpt":"We compare open- and closed-weight models using the Cyber ECI"},{"dimension":"summary","status":"uncertain","explanation":"The reading explains the Cyber ECI, and the definitions cover the ECI and benchmarks. However, the central contrast between open-weight and closed-weight models is never explained in plain language, and no term-library definition covers it. A general reader may not know that open-weight means the trained model parameters are publicly released. Smallest repair: add an open-weight / closed-weight term-library tooltip. The reading appropriately notes that the retrieved text does not substantiate the 6–7 month gap named in the title.","excerpt":"We compare open- and closed-weight models using the Cyber ECI"},{"dimension":"date","status":"pass","explanation":"The reading states October 16, 2026, which matches the UTC day of originalPublished. It uses no later events or hindsight framing. The CSV note dated Oct. 8, 2026 precedes publication, and the reading does not rely on it. The evidence kind is publisher-page, so no archive disclosure is needed.","excerpt":null},{"dimension":"support","status":"pass","explanation":"All three claims quote the source text verbatim and stay attributed. The caveat that the retrieved text lacks the results, and that a benchmark measure is not evidence of real-world cybersecurity performance, is a faithful statement of a gap rather than an invented claim. No unsupported benefits or costs are implied.","excerpt":"Scores are on the same scale as the general ECI."},{"dimension":"media","status":"pass","explanation":"No media is selected and no verified media candidates were supplied, so the null selection is appropriate.","excerpt":null}],"createdAt":"2026-11-04T21:43:50.961Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}