Anthropic Says Your AI Safety Score May Be Wrong by 10 Points

Anthropic published guidance on November 13, 2026, showing that confidence intervals on multi-turn AI safety evaluations can exceed ±10 percentage points — wide enough that a 94%-safe model and a 98%-safe model may be statistically indistinguishable. The company reported aggregate harmless-response rates of 94% to 98% for every recent model on its single-turn API tests. Those saturated single-turn numbers now serve mainly as regression checks on known failure modes.

The document targets a problem practitioners know well: a safety evaluation measures a model's propensity to cause or contribute to harm — whether its response makes harm more or less likely, and how severe that harm could be — not a specific capability. Anthropic's central concern is construct validity, meaning whether a test captures the concept it intends to measure. Get that wrong and every downstream number is noise.

Why refusal rate fails on self-harm

Anthropic's most striking finding: refusing to engage in suicide and self-harm conversations can itself be a failure. The desired response offers support and directs the user toward human help. A model that simply declines looks "safe" on a refusal metric while failing the actual job. Anthropic's multi-turn evaluations instead measure an appropriate-response rate across the entire conversation, using risk-specific rubrics in which one bad turn fails the whole exchange.

Building these tests takes structure. A domain expert writes a specification for the simulated user's objectives, tactics, persona, situation, tone and conversational style. The first user turn stays fixed; a separate Claude model generates subsequent turns in response to the tested model. Each scenario runs repeatedly with different combinations of characteristics, and a grader model scores the complete transcript against the relevant rubric.

Varying dimensions independently matters. If every severe case also uses an urgent tone, you can't separate the effects of severity and tone. Anthropic also recommends building scenarios from observed behavior, public data, red-teaming transcripts, incident reports or expert interviews — not invented situations alone. Its single-turn suites cover 16 policy areas in seven languages across multiple product surfaces, with the evaluation environment matched to the deployment being tested.

Cyber evaluations need containment: sandboxes without internet access and real-time monitoring, so a run can be stopped if a model's actions could cause real-world harm.

What does a bad grader do to your numbers?

Automated graders create another measurement risk, because broken systems can keep producing plausible scores. Anthropic's safeguards include atomic yes-or-no criteria, explicit treatment of disclaimers and borderline cases, loud failure alerts, and checks for response-length, position and self-preference biases. Human reviewers should independently label a sample of transcripts so grader-to-expert agreement can be compared with agreement among experts themselves.

That human baseline determines the remedy. If experts agree with one another but not with the grader, the rubric or grader needs work. If experts disagree with each other, the evaluation may not have a stable answer to measure. The guidance also calls for preregistering metrics, comparisons and improvement thresholds before anyone sees results.

Sample sizes underpin the judgments. Anthropic uses floors of 100 items for complex multi-turn evaluations and 1,000 for simpler single-turn tests, plus repeated runs to absorb noisy, nondeterministic outputs. The romance-scam example on claude.ai makes the point concrete: Claude Opus 5 recorded a 98% appropriate-response rate and Opus 5.5 recorded 94%, but the difference was not statistically significant. The numerical gap alone cannot establish which model behaved more safely.

Deployment configuration changes the score

While reviewing Claude Fable 5.1, Anthropic found some responses implicitly validated self-harm as a coping strategy even when they discouraged self-harm and pointed toward human support. Re-running the evaluation with targeted system-prompt mitigations produced a 94% appropriate-response rate on claude.ai, versus 60% on the API. A 34-point spread from configuration alone means scores from different deployments cannot be treated as equivalent.

Reporting recommendations follow: disclose model versions, sampling settings, prompts, grader versions and dates, and evaluate continuously during fine-tuning rather than running a single pre-release check. Severe flaws in deployed systems should go to the developer first, allowing time to remediate before publication.

Anthropic's three stated priorities: define the construct before collecting data, validate the grader against human judgment, and report uncertainty. The so-what for anyone comparing safety claims: whether one model is safer than another depends on test design, deployment context and statistical power as much as on the headline score.