# Building strong safety evaluations [Canonical reading](https://escapemeridian.com/readings/building-strong-safety-evaluations-92064c0b) Anthropic’s post argues that safety evaluations shape consequential decisions about deployment, safeguards, and research, but that a score is only as trustworthy as the evaluation behind it. A safety evaluation should measure whether a model responds appropriately to a specific risk, not simply whether it refuses. In conversations about self-harm, for example, refusal can itself be a failure; in dual-use areas such as biology and cybersecurity, an appropriate response may help with a legitimate request while withholding operational specifics that could enable harm. The post’s central point is that evaluations can mismeasure behavior: results may depend on prompt format, graders may fail, and test scenarios may miss how risk develops across a conversation. Anthropic recommends matching design to the question, using domain and methods experts, testing realistic multi-turn scenarios, checking graders against human judgments, preregistering analyses, and reporting uncertainty and configuration. It also describes how Anthropic uses system prompts and continuous evaluations during model development. The post cautions that these practices do not make every evaluation reliable: scenarios remain simpler than real users, and scores can be noisy or saturated. It recommends treating evaluation as an ongoing process, with methods that can change as evaluation science develops. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source Anthropic · Article Original source date: Nov 13, 2026 (2026-11-13T00:00:00.000Z). Date basis: Date stated by the publisher. Added to Escape Meridian: 2026-11-13T21:03:24.036Z. First observed: 2026-11-13T20:28:46.020Z. Retrieved: 2026-11-13T21:02:24.034Z. - [Building strong safety evaluations](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Limits This is Anthropic’s account of its recommendations and practices; the post’s examples do not independently establish how well those practices work across other evaluation settings. Anthropic says simulated users remain simpler than real users and that user behavior shifts over time, so the scenarios may not capture all real-world interactions. The post reports that confidence intervals for multi-turn evaluations on the API can be wide, and that evaluation scores may reflect sampling noise. The open question about how well simulations capture changing real-user behavior is not resolved by the post. ## Open question How well do Anthropic’s simulated multi-turn scenarios capture the ways real users’ behavior shifts over time? ## Source-stated Anthropic says safety evaluations inform decisions about deployment, safeguards, and future safety work, and that safety claims depend on the evaluations behind them. > Safety evaluations inform some of the most consequential decisions we make about an AI model, including whether it’s ready to deploy, which safeguards ship alongside it, and what our next round of safety training and research should focus on. - [Supporting source](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Source-stated Anthropic argues that refusal rate alone can misrepresent safety: for users discussing self-harm, a refusal may fail to provide appropriate support. > When a user is in crisis, simply declining to engage is, itself, a failure; the ideal response engages the user with empathy and points them toward human support. - [Supporting source](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Source-stated For dual-use requests, Anthropic says an appropriate response may help with the legitimate purpose while withholding details that could enable harm. > Rather, it engages with the legitimate version of the request while withholding the operational specifics that would provide real uplift to the harmful version of the request. - [Supporting source](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Source-stated Anthropic says single-turn tests can miss risks that emerge through escalation and context accumulated across a conversation. > Humans extract harmful information from a model through gradual escalation, so a model that handles the first message well may respond inappropriately by the fifth. - [Supporting source](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Source-stated Anthropic recommends validating graders against human judgments and reporting uncertainty, among other practices for building safety evaluations. > If you only do three things, do these: write down the construct you want to measure before collecting data, validate your grader against human judgment before trusting any number it produces, and report uncertainty so readers can tell signal from noise. - [Supporting source](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Source-stated Anthropic reports that simulated users remain simpler than real users and that user behavior changes over time. > Simulated users, even those based on real scenarios, are still much simpler than real ones, and user behavior shifts over time - [Supporting source](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Source and contributor context Anthropic · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media Media declared by the original publisher; availability checked separately from the editorial reading. - [Illustration of two dark green gridded squares on a light green background. On the left, a tilted red square and an upright blue square overlap imperfectly, le… · Anthropic](https://assets.claude.com/bab5121b9ac9a4bc3d4f5a212714bb21736398e8.jpg?w=1920&q=75&fm=webp&fit=max) ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Confidence interval A range of values consistent with the data at a stated level of confidence. - [Definition reference](https://en.wikipedia.org/wiki/Confidence_interval) ## API An application programming interface: a defined way for software to request data or actions from another system. - [Definition reference](https://developer.mozilla.org/en-US/docs/Glossary/API) ## Anthropic Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems. - [Explore this context](/entities/anthropic) - [Reference](https://www.anthropic.com/news) - [Reference](https://www.anthropic.com/research/team/economics) ## TIME TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services. - [Explore this context](/entities/time) - [Reference](https://time.com/) ## Editorial provenance Version 1. Publication ID: 03348a00-0f2b-46cf-8059-1c67ee7aff36. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T21:03:24.036Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: useful\_perspective. Observation: 52b42be941c916deb7e54b4d7c2ff52e092781b727afbeee9f8f31889c7146ea. Source text hash: 4c906f72db0c97c20f67761ac9a7537de90cc264c563877e68175dac9146e8e6. ## Revision and source-check history ### Version 1 — 2026-11-13T21:03:24.036Z Anthropic’s post argues that safety evaluations shape consequential decisions about deployment, safeguards, and research, but that a score is only as trustworthy as the evaluation behind it. A safety evaluation should measure whether a model responds appropriately to a specific risk, not simply whether it refuses. In conversations about self-harm, for example, refusal can itself be a failure; in dual-use areas such as biology and cybersecurity, an appropriate response may help with a legitimate request while withholding operational specifics that could enable harm. The post’s central point is that evaluations can mismeasure behavior: results may depend on prompt format, graders may fail, and test scenarios may miss how risk develops across a conversation. Anthropic recommends matching design to the question, using domain and methods experts, testing realistic multi-turn scenarios, checking graders against human judgments, preregistering analyses, and reporting uncertainty and configuration. It also describes how Anthropic uses system prompts and continuous evaluations during model development. The post cautions that these practices do not make every evaluation reliable: scenarios remain simpler than real users, and scores can be noisy or saturated. It recommends treating evaluation as an ongoing process, with methods that can change as evaluation science develops. ## Original source Anthropic · Article Original source date: Nov 13, 2026 (2026-11-13T00:00:00.000Z). Date basis: Date stated by the publisher. Added to Escape Meridian: 2026-11-13T21:03:24.036Z. First observed: 2026-11-13T20:28:46.020Z. Retrieved: 2026-11-13T21:02:24.034Z. - [Building strong safety evaluations](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Limits This is Anthropic’s account of its recommendations and practices; the post’s examples do not independently establish how well those practices work across other evaluation settings. Anthropic says simulated users remain simpler than real users and that user behavior shifts over time, so the scenarios may not capture all real-world interactions. The post reports that confidence intervals for multi-turn evaluations on the API can be wide, and that evaluation scores may reflect sampling noise. The open question about how well simulations capture changing real-user behavior is not resolved by the post. ## Open question How well do Anthropic’s simulated multi-turn scenarios capture the ways real users’ behavior shifts over time? ## Source-stated Anthropic says safety evaluations inform decisions about deployment, safeguards, and future safety work, and that safety claims depend on the evaluations behind them. > Safety evaluations inform some of the most consequential decisions we make about an AI model, including whether it’s ready to deploy, which safeguards ship alongside it, and what our next round of safety training and research should focus on. - [Supporting source](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Source-stated Anthropic argues that refusal rate alone can misrepresent safety: for users discussing self-harm, a refusal may fail to provide appropriate support. > When a user is in crisis, simply declining to engage is, itself, a failure; the ideal response engages the user with empathy and points them toward human support. - [Supporting source](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Source-stated For dual-use requests, Anthropic says an appropriate response may help with the legitimate purpose while withholding details that could enable harm. > Rather, it engages with the legitimate version of the request while withholding the operational specifics that would provide real uplift to the harmful version of the request. - [Supporting source](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Source-stated Anthropic says single-turn tests can miss risks that emerge through escalation and context accumulated across a conversation. > Humans extract harmful information from a model through gradual escalation, so a model that handles the first message well may respond inappropriately by the fifth. - [Supporting source](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Source-stated Anthropic recommends validating graders against human judgments and reporting uncertainty, among other practices for building safety evaluations. > If you only do three things, do these: write down the construct you want to measure before collecting data, validate your grader against human judgment before trusting any number it produces, and report uncertainty so readers can tell signal from noise. - [Supporting source](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Source-stated Anthropic reports that simulated users remain simpler than real users and that user behavior changes over time. > Simulated users, even those based on real scenarios, are still much simpler than real ones, and user behavior shifts over time - [Supporting source](https://www.anthropic.com/research/building-strong-safety-evaluations) ## Source and contributor context Anthropic · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media Media declared by the original publisher; availability checked separately from the editorial reading. - [Illustration of two dark green gridded squares on a light green background. On the left, a tilted red square and an upright blue square overlap imperfectly, le… · Anthropic](https://assets.claude.com/bab5121b9ac9a4bc3d4f5a212714bb21736398e8.jpg?w=1920&q=75&fm=webp&fit=max) ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Confidence interval A range of values consistent with the data at a stated level of confidence. - [Definition reference](https://en.wikipedia.org/wiki/Confidence_interval) ## API An application programming interface: a defined way for software to request data or actions from another system. - [Definition reference](https://developer.mozilla.org/en-US/docs/Glossary/API) ## Anthropic Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems. - [Explore this context](/entities/anthropic) - [Reference](https://www.anthropic.com/news) - [Reference](https://www.anthropic.com/research/team/economics) ## TIME TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services. - [Explore this context](/entities/time) - [Reference](https://time.com/) ## Editorial provenance Version 1. Publication ID: 03348a00-0f2b-46cf-8059-1c67ee7aff36. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T21:03:24.036Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: useful\_perspective. Observation: 52b42be941c916deb7e54b4d7c2ff52e092781b727afbeee9f8f31889c7146ea. Source text hash: 4c906f72db0c97c20f67761ac9a7537de90cc264c563877e68175dac9146e8e6. ### uncertain — 2026-11-13T21:03:24.036Z — anthropic/claude-opus-5.5 Publication: 03348a00-0f2b-46cf-8059-1c67ee7aff36 attribution — pass: The recommendations, examples and limitations are consistently attributed to Anthropic's post on its own research page. The reading does not present Anthropic's account as established fact or as Escape Meridian's view. > this post shares recommendations for anyone who builds or relies on AI safety evaluations, using examples from our system cards summary — uncertain: The reading is mostly faithful, but several central terms are never explained in plain language: graders, saturated, preregistering, system prompts and multi-turn. A non-specialist will not know that a grader is itself an AI model scoring transcripts. The second paragraph also turns the argument into a list. It drops the post's concrete evidence, including the 34-point API vs claude.ai gap, the self-harm fix (60% vs 94%) and the Opus 5 vs 5.5 non-significant difference. Smallest repair: briefly gloss these terms and restore one concrete example. > Most modern evals use another AI model to review and grade responses, often called an LLM judge or grader. date — pass: The originalPublished date of 2026-11-13 comes from the publisher page and matches the feed entry. No earlier publication is indicated, and the draft makes no conflicting date claim. support — pass: Every claim excerpt is an exact quote and supports its claim. The question about how well simulated scenarios track shifting real-user behavior follows directly from the source's own caveat. The limitations about wide confidence intervals and sampling noise are source-stated. The framing that the practices do not make every evaluation reliable is a mild paraphrase, but the stated caveats support it. > Simulated users, even those based on real scenarios, are still much simpler than real ones, and user behavior shifts over time media — pass: The selected media is the verified direct link to the original Anthropic article, which is the most direct way to reach the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/92064c0b2a94465a4ba9a0ac45a9cc84d7142e43367f87f05b148170c77d5afa/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/building-strong-safety-evaluations-92064c0b.json)