METR’s Anthropic Assessment: What AI R&D Acceleration Means

METR’s November 5 assessment, by Tom Cunningham, Nate Rush, Parker Whitfill and Hjalmar Wijk, estimates that the rate of progress on AECI, Anthropic’s aggregate capability index, became 1.9 times its earlier rate after October 2025. The 95% confidence interval is 1.5–2.8; alternative assumptions yield 1.1–2.9.

The authors subjectively attribute most current October acceleration to AI, perhaps 80%. This uncertain share is not an 80% probability. Anthropic redacted much crucial evidence, making the judgment hard to scrutinize. The public web report summarizes a roughly 100-page private report; it does not provide that unredacted document. The interval excludes uncertainty over which models enter the analysis.

What does 1.9 times faster mean?

Imagine a fictional capability index that rose by 10 points a year before a change and 19 points a year afterward. The later rate is 1.9 times the earlier rate. This example is invented to explain the arithmetic; it is not Anthropic data. It does not mean the system became 1.9 times as intelligent, nor that a 19-point improvement has the same practical value at every point on the scale.

The distinction between a rate and a level matters. A faster rate says the line is getting steeper under a particular way of measuring progress. It does not tell us how much useful work a model can do outside the benchmark, what an individual researcher saved in time, or how long the faster trend will last.

A separate hypothetical illustrates why a task-level speedup cannot simply be pasted onto a whole research program. Suppose implementation and review each initially take half a project’s time. If implementation becomes four times faster and review takes the same time as before, the total falls from one unit of time to 0.625 units. The project is 1.6 times faster overall, not four times faster. Real research workflows have many more moving parts, and their proportions can change.

How do you separate AI from everything else?

Before this assessment, METR published a proposal for evaluating AI R&D acceleration. It separates measuring progress from attributing its cause, identifying constraints, and checking whether the assessment could detect a much larger change. It recommends looking across benchmarks, internal indicators and interviews, while considering changes in compute, staffing, data and algorithms.

The hard part is the missing comparison. Researchers cannot directly observe the same organization, at the same moment, with the same people and resources but without AI assistance. They have to estimate that alternative using incomplete evidence. Interviews can explain what happened inside a team, but a person’s impression of time saved does not by itself establish the counterfactual output of the whole organization.

There are also three different kinds of uncertainty here. A confidence interval describes variation within a specified statistical model. A range produced by changing model assumptions asks how the answer moves when the analytical lens changes. Attribution adds another question: how much of the observed change would have happened without AI? The first number cannot settle the other two.

Automation is a different measure

Anthropic has published its own prototype index of AI’s role in development tasks, by Marina Favaro and Phillie Wright, with research direction from Jack Clark. For August 2026, the company reported that AI led 26% of measured R&D work and collaborated or did more in over 90%; it did not classify any measured category as fully autonomous. Those are company-reported task classifications, not percentages of productivity growth or METR’s October acceleration estimate.

The task basket drew on sampled July work records and was assembled and rated using Claude. It was weighted by human time and held fixed for comparisons. Anthropic identifies limitations including rating disagreements, correlated model errors and new work outside the fixed basket. The index can illuminate changes in how work is done, but its categories need an additional link to establish how much useful research output changed.

That difference affects interpretation. A team could automate a frequent administrative step with little effect on the pace of its hardest experiments. Conversely, help on a rare bottleneck could matter disproportionately. Counts of tasks, hours, code or model actions answer different questions from the pace of validated research progress.

The separate July risk review

METR published a companion review of Anthropic’s August risk report on November 5, also credited to Cunningham, Rush, Whitfill and Wijk. It concerns models available on July 15, not the later October assessment. The reviewers broadly agreed that those models could not fully replace Anthropic’s researchers and that direct catastrophic risks from automated R&D were low at that time.

They questioned the confidence with which the public risk report said an acceleration warning threshold had not been crossed. Most relevant researchers leaned against crossing it, while METR considered crossing plausible. METR calculated 1.97 times acceleration from private-report estimates and argued that describing it as around twice would better convey the evidence than less than twice. This concerns a warning threshold and uncertainty, not a finding that catastrophic risk was already high.

Threshold language compresses a continuous and uncertain estimate into a yes-or-no decision. An estimate near a boundary can be useful for deciding what to investigate next while still leaving room for reasonable disagreement about whether the boundary was crossed. Before comparing two estimates, check their measurement dates, definitions and units; numerical similarity alone does not make them interchangeable.

Access, redactions and independence

Anthropic separately announced an evaluator-access arrangement on October 15. It said six METR employees had received company hardware, office space and access comparable to internal risk-assessment teams, subject to exclusions and specified confidential-information redactions. Anthropic determines which information falls into the redaction categories. METR says the acceleration assessment used a different arrangement, without employee-like access.

Access and public inspectability answer different questions. Access can let an outside team examine evidence unavailable to ordinary readers. Redactions can then prevent those readers from checking the analysis in full. An evaluator’s access and a reader’s ability to reproduce a finding should therefore be assessed separately.

Assistance and full autonomy

Anthropic’s separate discussion of recursive self-improvement, by Marina Favaro and Jack Clark, distinguishes growing human-directed assistance from a system autonomously designing and building a successor. The company also discusses possible constraints such as research judgment, hardware, energy and organizational capacity. It presents its own view; it is not a response to METR’s November assessment.

A useful question for future assessments is whether they can connect changes in AI assistance to independently inspectable measures of research output, while making clear which evidence remains private and which assumptions carry the result. Readers should also ask whether a proposed constraint would limit the present workflow, a more automated workflow, or both. Those are different forecasting problems.

Sources

• METR: Independent Assessment of AI R&D Acceleration at Anthropic in October 2026.
• Tom Cunningham, Nate Rush, Parker Whitfill and Hjalmar Wijk, METR: Review of the “Risks from automated R&D” section in Anthropic’s Risk Report (August 2026).
• METR: How an assessor could investigate evidence of acceleration from the automation of AI R&D.
• Marina Favaro and Phillie Wright, research direction by Jack Clark, Anthropic: Measurements for understanding the pace of AI development inside frontier labs.
• Anthropic: Anthropic update on independent evaluators.
• Marina Favaro and Jack Clark, Anthropic: When AI builds itself.

Revision log

November 5, 2026: Published.