· Jul 1, 2024
Anthropic outlines funding for third-party model evaluations
Markdown · JSON · Original source
’s post, published July 1, 2024, announces funding for third-party evaluations of advanced AI capabilities. Anthropic says the supply of high-quality, safety-relevant evaluations does not meet demand, and argues that outside evaluation work can provide tools for the wider field of AI safety. The proposed work ranges from assessments for AI Safety Levels to evaluations of advanced science, societal impacts, and model autonomy. Anthropic also seeks infrastructure to help subject-matter experts develop evaluations, and trials comparing task performance between groups with and without model access. The post’s central concern is whether evaluations measure the risks they are meant to capture: Anthropic says even strong performance on an evaluation may not be enough to worry experts. It calls for realistic, safety-relevant threat modeling and presents the initiative as a way to advance comprehensive AI evaluation as an industry standard. The post does not establish whether funded evaluations will track risks beyond the tests themselves.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
· Article
Source details
Original source date: Jul 1, 2024 (2024-07-01T00:00:00.000Z).
Date basis: Date stated on the publisher’s page.
Added to Escape Meridian: 2026-11-18T12:08:16.089Z. First observed: 2026-10-21T23:08:20.413Z. Retrieved: 2026-10-24T11:13:35.541Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
Text as retrieved on October 24, 2026; originally published July 1, 2024. says even strong evaluation results may not indicate the risks the evaluations are intended to measure. The post does not report outcomes from the funded evaluations.
Open question
How will funded evaluations show that their results track risks outside the test itself?
Source-stated
Anthropic announces an initiative to fund evaluations developed by third-party organizations to measure advanced AI capabilities.
today we're introducing a new initiative to fund evaluations developed by third-party organizations that can effectively measure advanced capabilities in AI models.
Source-stated
Anthropic says demand for high-quality, safety-relevant evaluations exceeds supply.
Developing high-quality, safety-relevant evaluations remains challenging, and the demand is outpacing the supply.
Source-stated
Anthropic says evaluation performance may not adequately indicate the risks an evaluation is intended to measure.
even the best evaluations are not always indicative of risks they purport to measure.
Source-stated
Anthropic presents the initiative as supporting progress toward making comprehensive AI evaluation an industry standard.
We hope this initiative serves as a catalyst for progress towards a future where comprehensive AI evaluation is an industry standard.
Source-stated
Anthropic says that high performance on an evaluation has often not been sufficient to concern experts.
Most of the time, when models have performed highly, experts have realized that high performance on that version of the evaluation is not sufficient to worry them.
Source and contributor context
Anthropic · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Evals
Structured tests of what an AI system can do or how it behaves, run before or after release.
Anthropic
Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Editorial provenance
Version 1. Publication ID: 411350ce-8bf1-2cac-a4e4-21cf3970780c.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-18T12:03:46.933Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.
Observation: a10c4ff88afcb6aeacdc0a451794d8579d7ea518d085e162793379599be38efb. Source text hash: b6be36431294a996764d88889f7faadd3651c1529cc64d01621f78cbf1c3aa1e.
Revision and source-check history
Version 1 · 2026-11-18T12:08:16.089Z
Anthropic outlines funding for third-party model evaluations
’s post, published July 1, 2024, announces funding for third-party evaluations of advanced AI capabilities. Anthropic says the supply of high-quality, safety-relevant evaluations does not meet demand, and argues that outside evaluation work can provide tools for the wider field of AI safety. The proposed work ranges from assessments for AI Safety Levels to evaluations of advanced science, societal impacts, and model autonomy. Anthropic also seeks infrastructure to help subject-matter experts develop evaluations, and trials comparing task performance between groups with and without model access. The post’s central concern is whether evaluations measure the risks they are meant to capture: Anthropic says even strong performance on an evaluation may not be enough to worry experts. It calls for realistic, safety-relevant threat modeling and presents the initiative as a way to advance comprehensive AI evaluation as an industry standard. The post does not establish whether funded evaluations will track risks beyond the tests themselves.
Original source
· Article
Source details
Original source date: Jul 1, 2024 (2024-07-01T00:00:00.000Z).
Date basis: Date stated on the publisher’s page.
Added to Escape Meridian: 2026-11-18T12:08:16.089Z. First observed: 2026-10-21T23:08:20.413Z. Retrieved: 2026-10-24T11:13:35.541Z.
Limits
Text as retrieved on October 24, 2026; originally published July 1, 2024. says even strong evaluation results may not indicate the risks the evaluations are intended to measure. The post does not report outcomes from the funded evaluations.
Open question
How will funded evaluations show that their results track risks outside the test itself?
Source-stated
Anthropic announces an initiative to fund evaluations developed by third-party organizations to measure advanced AI capabilities.
today we're introducing a new initiative to fund evaluations developed by third-party organizations that can effectively measure advanced capabilities in AI models.
Source-stated
Anthropic says demand for high-quality, safety-relevant evaluations exceeds supply.
Developing high-quality, safety-relevant evaluations remains challenging, and the demand is outpacing the supply.
Source-stated
Anthropic says evaluation performance may not adequately indicate the risks an evaluation is intended to measure.
even the best evaluations are not always indicative of risks they purport to measure.
Source-stated
Anthropic presents the initiative as supporting progress toward making comprehensive AI evaluation an industry standard.
We hope this initiative serves as a catalyst for progress towards a future where comprehensive AI evaluation is an industry standard.
Source-stated
Anthropic says that high performance on an evaluation has often not been sufficient to concern experts.
Most of the time, when models have performed highly, experts have realized that high performance on that version of the evaluation is not sufficient to worry them.
Source and contributor context
Anthropic · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Evals
Structured tests of what an AI system can do or how it behaves, run before or after release.
Anthropic
Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Editorial provenance
Version 1. Publication ID: 411350ce-8bf1-2cac-a4e4-21cf3970780c.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-18T12:03:46.933Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.
Observation: a10c4ff88afcb6aeacdc0a451794d8579d7ea518d085e162793379599be38efb. Source text hash: b6be36431294a996764d88889f7faadd3651c1529cc64d01621f78cbf1c3aa1e.
uncertain · 2026-11-18T12:03:46.933Z · anthropic/claude-opus-5.5
Publication: 411350ce-8bf1-2cac-a4e4-21cf3970780c
attribution — pass: The reading consistently attributes the announcement, the supply-demand claim and the evaluation principles to Anthropic, the publisher of this single post. It does not present Anthropic's assertions as established fact.
today we're introducing a new initiative to fund evaluations developed by third-party organizations
summary — uncertain: "AI Safety Levels" is a central term that is neither explained in the reading nor in its definitions. A general reader cannot tell that these are levels from Anthropic's Responsible Scaling Policy that set safety and security requirements. Smallest repair: add one clause saying so. The reading also calls risk-tracking "the post's central concern", but the source makes that point as one of ten principles within a funding call. Smallest repair: call it a key principle rather than the central concern. The title and question are otherwise fair.
These levels determine the safety and security requirements for models with specific capabilities.
date — pass: The stated July 1, 2024 matches the UTC day of originalPublished, and there is no hindsight framing. The page was modified in October 2026. The cited passages are core body text consistent with the original announcement, and the reading relies on no apparent later addition. Evidence kind is publisher-page, so no archive disclosure is required.
support — pass: Each claim matches its excerpt verbatim. The summary's remark that outside work can provide tools for the field rests on the opening paragraph. The question and limitation stay within the source and add no unsupported benefits or costs. The later page edit does not appear to touch these core passages.
Our investment in these evaluations is intended to elevate the entire field of AI safety, providing valuable tools that benefit the whole ecosystem.
media — pass: No media was selected, and no verified media candidates were supplied, so nothing is misidentified.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Explore AnthropicAnthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Explore AnthropicAnthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Explore AnthropicAnthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Explore AnthropicAnthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Explore AnthropicAnthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Explore AnthropicAnthropic develops Claude and publishes research about AI capability, safety and how people use its systems.
Explore Anthropic