METR Says Anthropic's AI Research Pace Is Nearly 2x Faster

By Olivia Hart

METR estimated on November 5 that, since October 2025, Anthropic’s models improved on benchmarks at 1.9 times their pre-October 2025 rate. The 95% confidence interval ran from 1.5x to 2.8x, leaving substantial room for either faster or slower progress.

METR’s 11-week, high-access assessment at Anthropic ran from August 12 to October 29, 2026. The independent AI safety organization reviewed internal benchmark scores, training-compute data, algorithmic-progress estimates and code contributions, and interviewed roughly 15 Anthropic researchers.

METR described the result as roughly two years of historical model progress in the past year. It compared the gain with the distance from Claude 2.1 to Sonnet 4.5, a benchmark measure rather than proof of matching gains in the real world.

What does the 1.9x estimate measure?

METR used AECI, Anthropic’s internal version of the Epoch Capabilities Index, which aggregates scores across multiple benchmarks. Robustness checks that changed benchmark selection, model dating and statistical assumptions produced estimates from 1.1x to 2.9x.

METR wrote: “We would not be surprised if the true acceleration was closer to 1.3x or 3x the historical rate.” That caveat sets the practical boundary for reading a single headline multiplier as precision.

METR attributed approximately 80% of the acceleration to Anthropic’s extensive use of AI in research and development, while warning of substantial uncertainty. Its supporting evidence pointed in three directions:

• Researcher interviews centered on roughly 2x speedups in their areas.
• Internal Slack polls showed self-reported uplift as high as 12x by October 2026; METR called that only very loose evidence.
• METR cited signs of life suggesting fully autonomous AI contributions could grow soon. It said AI currently assists rather than leads Anthropic’s most important training work.

Takeaway: METR found a strong acceleration signal, but its own evidence supports a range rather than a definitive multiplier.

Did faster compute drive the gain?

METR said it received convincing evidence that faster growth in training compute did not drive the acceleration. Anthropic also redacted those supporting details, leaving outside readers unable to inspect the basis for that conclusion.

The assessment found no clear bottleneck in algorithmic progress, training compute, experiment compute or data that would constrain Anthropic over the next 12 months. METR also said its understanding of bottlenecks changed several times as it received more detailed information and that it lacked confidence in its current assessment.

Takeaway: METR projects no immediate ceiling on Anthropic’s progress, yet it cannot show publicly why it reached that forecast.

Why did Anthropic withhold the evidence?

Anthropic redacted a majority of the evidence METR considered most informative. Anthropic said the redactions were “aimed at removing confidential information that could accelerate other AI developers.”

The withheld material included detailed acceleration estimates for specific pipeline stages, algorithmic-progress breakdowns, bottleneck analyses and forecasts for future models. METR said the public therefore had to rely more heavily on METR’s judgment than it ideally should.

The original engagement included verification of how Anthropic allocated compute between capabilities and safety research. Anthropic canceled that component midway, saying it wanted to keep the work focused on risk assessment. METR said it worked on Anthropic devices, used Anthropic-provided tokens and took no payment.

METR retained unilateral rights to disclose the existence of redactions and whether findings depended on them. The cited materials do not specify whether any published finding changed because of the withheld material.

Takeaway: The access arrangement gave METR unusual reach, but Anthropic’s secrecy still prevents a public audit of the report’s most probative evidence.

What does the finding mean for safety thresholds?

Anthropic’s Responsible Scaling Policy lists AI R&D acceleration as a key risk indicator, METR said. In a personal EA Forum analysis, a METR colleague described the policy’s threshold for dramatically accelerated progress due to AI as observing or expecting overall capabilities progress at twice the fastest sustained rate before AI contributed significantly to R&D, plausibly attributable to AI automation.

The colleague argued that threshold has likely been crossed. The analysis represents one METR colleague’s interpretation, not an Anthropic determination under its policy. That distinction matters because the same 1.9x estimate can trigger scrutiny without proving that every condition of the threshold has been met.

Takeaway: METR’s finding sits near the policy threshold, but the threshold analysis reaches its conclusion by inference.

How much weight can the estimate bear?

METR listed three conditions that could push acceleration higher: recent growth could keep increasing, possibly hyperbolically; automation could be closer than estimated; or AECI could understate real-world impact. METR cited sudden threshold effects, including the prospect of resolving Millennium Prize Problems, as an example of what a benchmark index might miss.

The estimate could fall if non-AI factors explain most gains, progress remains limited to well-specified benchmarks, or Anthropic’s 2023–2025 historical rate was unusually low. Including non-Anthropic frontier models lowered the acceleration estimate to roughly 1.4x. METR said that result suggested Anthropic may have been subfrontier in 2025.

Takeaway: The broader model set already cuts the estimate from 1.9x to about 1.4x, which makes METR’s central figure conditional rather than settled.

What else did Anthropic disclose?

During the assessment, Anthropic CEO Dario Amodei published a mid-September essay titled We Must Pace the Frontier. The Verge, cited alongside METR’s assessment, reported that Anthropic was also preparing to go public.

Anthropic’s own R&D Automation Index shows Claude leading 26% of the company’s AI R&D work. Anthropic also reports that the share of work at or above the AI-collaborates level exceeds 90%. METR said its assessment used a different methodology and deeper access.

Anthropic has established three tiers of evaluator access. METR operated at the highest tier, which Anthropic describes as employee-like access, as frontier labs increasingly opened themselves to outside evaluators.

Takeaway: METR’s access model still leaves commercial secrecy in tension with public accountability, and the next test is whether future evaluations can narrow the 1.1x–2.9x range without relying on Anthropic’s withheld forecasts.