# Epoch AI describes comparing open- and closed-weight models on cybersecurity benchmarks [Canonical reading](https://escapemeridian.com/readings/epoch-ai-describes-comparing-open-and-closed-weight-models-on-dbbe08f6) Epoch AI’s page was published on October 16, 2026. It describes comparing open- and closed-weight models with the Cyber ECI, a cybersecurity-specific version of the Epoch Capabilities Index (ECI). Epoch AI says it keeps benchmark parameters from a joint fit using general and cyber benchmarks, then estimates each model’s Cyber ECI from its cybersecurity results alone; scores use the same scale as the general ECI. The page links to data, analysis, assumptions and limitations, and methodology documentation, but the retrieved text does not include the results or explain the size of the gap named in the page title. The retrieved text therefore leaves unknown what Cyber ECI scores and analysis support the stated comparison between open- and closed-weight models. The comparison describes a benchmark-based measure, not evidence here about models’ performance in real-world cybersecurity work. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source Epoch AI · Article Original source date: Oct 16, 2026 (2026-10-16T00:00:00.000Z). Date basis: Date supplied by the publisher’s feed. Added to Escape Meridian: 2026-11-04T21:48:27.563Z. First observed: 2026-10-18T02:34:56.924Z. Retrieved: 2026-10-29T12:05:44.669Z. - [Open-weight models trail closed models in cyber capabilities by 6–7 months](https://epoch.ai/data-insights/open-closed-cyber-eci-gap) ## Limits Text as retrieved on October 29, 2026; originally published October 16, 2026. The retrieved page text does not include the data, analysis, or methodology details, so it does not substantiate the size of the comparison named in the page title or show how the scores relate to real-world cybersecurity performance. ## Open question What Cyber ECI scores and analysis support the stated comparison between open- and closed-weight models? ## Source-stated Epoch AI says it compares open- and closed-weight models using the Cyber ECI, a domain-specific version of the Epoch Capabilities Index based on cybersecurity benchmarks. > We compare open- and closed-weight models using the Cyber ECI, a domain-specific version of the Epoch Capabilities Index (ECI) based on cybersecurity benchmarks. - [Supporting source](https://epoch.ai/data-insights/open-closed-cyber-eci-gap) ## Source-stated Epoch AI says it estimates each model’s Cyber ECI from cyber results alone, while retaining benchmark parameters from a joint fit on general and cyber benchmarks. > We keep the benchmark parameters from a joint fit of the ECI on general and cyber benchmarks, and estimate each model’s Cyber ECI from its cyber results alone. - [Supporting source](https://epoch.ai/data-insights/open-closed-cyber-eci-gap) ## Source-stated Epoch AI says Cyber ECI scores use the same scale as the general ECI. > Scores are on the same scale as the general ECI. - [Supporting source](https://epoch.ai/data-insights/open-closed-cyber-eci-gap) ## Source and contributor context Epoch AI · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## Epoch Capabilities Index Epoch AI's index that combines scores from many AI benchmarks into a single general-capability scale, so models can be compared even over periods long enough for single benchmarks to saturate. - [Definition reference](https://epoch.ai/data/eci-documentation) ## Epoch AI Epoch AI studies trends in AI capability, training resources and the infrastructure behind them. - [Explore this context](/entities/epoch-ai) - [Reference](https://epoch.ai/) - [Reference](https://epoch.ai/data) ## Editorial provenance Version 1. Publication ID: 3f3b1987-9724-ce0b-96b0-3866e5b88d9b. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-04T21:43:50.961Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model, second\_opinion\_categorical. Observation: caaf879387d498f2fd9e897b7e63f5ccc8fea88379216062f66ea8ddfff653ff. Source text hash: a0c28864e34539d63f7a87f8966781dce04d30688729e2e91ec8dd2720ee7838. ## Revision and source-check history ### Version 1 — 2026-11-04T21:48:27.563Z Epoch AI’s page was published on October 16, 2026. It describes comparing open- and closed-weight models with the Cyber ECI, a cybersecurity-specific version of the Epoch Capabilities Index (ECI). Epoch AI says it keeps benchmark parameters from a joint fit using general and cyber benchmarks, then estimates each model’s Cyber ECI from its cybersecurity results alone; scores use the same scale as the general ECI. The page links to data, analysis, assumptions and limitations, and methodology documentation, but the retrieved text does not include the results or explain the size of the gap named in the page title. The retrieved text therefore leaves unknown what Cyber ECI scores and analysis support the stated comparison between open- and closed-weight models. The comparison describes a benchmark-based measure, not evidence here about models’ performance in real-world cybersecurity work. ## Original source Epoch AI · Article Original source date: Oct 16, 2026 (2026-10-16T00:00:00.000Z). Date basis: Date supplied by the publisher’s feed. Added to Escape Meridian: 2026-11-04T21:48:27.563Z. First observed: 2026-10-18T02:34:56.924Z. Retrieved: 2026-10-29T12:05:44.669Z. - [Open-weight models trail closed models in cyber capabilities by 6–7 months](https://epoch.ai/data-insights/open-closed-cyber-eci-gap) ## Limits Text as retrieved on October 29, 2026; originally published October 16, 2026. The retrieved page text does not include the data, analysis, or methodology details, so it does not substantiate the size of the comparison named in the page title or show how the scores relate to real-world cybersecurity performance. ## Open question What Cyber ECI scores and analysis support the stated comparison between open- and closed-weight models? ## Source-stated Epoch AI says it compares open- and closed-weight models using the Cyber ECI, a domain-specific version of the Epoch Capabilities Index based on cybersecurity benchmarks. > We compare open- and closed-weight models using the Cyber ECI, a domain-specific version of the Epoch Capabilities Index (ECI) based on cybersecurity benchmarks. - [Supporting source](https://epoch.ai/data-insights/open-closed-cyber-eci-gap) ## Source-stated Epoch AI says it estimates each model’s Cyber ECI from cyber results alone, while retaining benchmark parameters from a joint fit on general and cyber benchmarks. > We keep the benchmark parameters from a joint fit of the ECI on general and cyber benchmarks, and estimate each model’s Cyber ECI from its cyber results alone. - [Supporting source](https://epoch.ai/data-insights/open-closed-cyber-eci-gap) ## Source-stated Epoch AI says Cyber ECI scores use the same scale as the general ECI. > Scores are on the same scale as the general ECI. - [Supporting source](https://epoch.ai/data-insights/open-closed-cyber-eci-gap) ## Source and contributor context Epoch AI · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## Epoch Capabilities Index Epoch AI's index that combines scores from many AI benchmarks into a single general-capability scale, so models can be compared even over periods long enough for single benchmarks to saturate. - [Definition reference](https://epoch.ai/data/eci-documentation) ## Epoch AI Epoch AI studies trends in AI capability, training resources and the infrastructure behind them. - [Explore this context](/entities/epoch-ai) - [Reference](https://epoch.ai/) - [Reference](https://epoch.ai/data) ## Editorial provenance Version 1. Publication ID: 3f3b1987-9724-ce0b-96b0-3866e5b88d9b. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-04T21:43:50.961Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model, second\_opinion\_categorical. Observation: caaf879387d498f2fd9e897b7e63f5ccc8fea88379216062f66ea8ddfff653ff. Source text hash: a0c28864e34539d63f7a87f8966781dce04d30688729e2e91ec8dd2720ee7838. ### uncertain — 2026-11-04T21:43:50.961Z — anthropic/claude-opus-5.5 Publication: 3f3b1987-9724-ce0b-96b0-3866e5b88d9b attribution — pass: The page is a single Epoch AI data insight describing its own Cyber ECI comparison, not a listing of other pieces. The reading attributes the methodology statements to Epoch AI, which matches the publisher and the first-person text. > We compare open- and closed-weight models using the Cyber ECI summary — uncertain: The reading explains the Cyber ECI, and the definitions cover the ECI and benchmarks. However, the central contrast between open-weight and closed-weight models is never explained in plain language, and no term-library definition covers it. A general reader may not know that open-weight means the trained model parameters are publicly released. Smallest repair: add an open-weight / closed-weight term-library tooltip. The reading appropriately notes that the retrieved text does not substantiate the 6–7 month gap named in the title. > We compare open- and closed-weight models using the Cyber ECI date — pass: The reading states October 16, 2026, which matches the UTC day of originalPublished. It uses no later events or hindsight framing. The CSV note dated Oct. 8, 2026 precedes publication, and the reading does not rely on it. The evidence kind is publisher-page, so no archive disclosure is needed. support — pass: All three claims quote the source text verbatim and stay attributed. The caveat that the retrieved text lacks the results, and that a benchmark measure is not evidence of real-world cybersecurity performance, is a faithful statement of a gap rather than an invented claim. No unsupported benefits or costs are implied. > Scores are on the same scale as the general ECI. media — pass: No media is selected and no verified media candidates were supplied, so the null selection is appropriate. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/dbbe08f61c97553b6d2458312ec4185da814bf03dc0baba9cfa147ecce1eb220/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/epoch-ai-describes-comparing-open-and-closed-weight-models-on-dbbe08f6.json)