Kingy AI
Subscribe, 2.1M subscribersWork with Kingy
AI News

Anthropic’s RSI Warning: Are We Closer to AI That Improves Itself?

November 2, 2026 Blog
Conceptual loop from AI assistance through human research and validation to more capable AI, with potential feedback into research

Anthropic forecasts that future models may soon meet its AI-research acceleration threshold. It has not designated a current model above it.

Faster research and autonomous successor-building would both be consequential, but they describe different things. A lab could experience a large increase in research progress while people still set objectives, review results, allocate computing resources, and decide which models to train. A system that autonomously designs and develops its successor would go further.

In its November 2 amended IPO filing, internal estimates and “expectations regarding future models, rather than our current models” suggest it “may imminently meet” that threshold. The passage supplies no model identity, probability, supporting time series, or definition of imminent.

My reading: the warning increases the urgency of preparing for faster AI research. Public evidence still cannot establish when a fully autonomous self-improvement loop will arrive. The useful questions concern what AI contributes to research, which bottlenecks remain, and whether safety work can keep pace.

Four different things can be happening

The phrase recursive self-improvement often compresses several steps into one dramatic idea. Separating them makes the filing easier to understand. The following is an editorial framework, not an official Anthropic classification.

Level of claim What it means What it does not establish
AI assistance A researcher uses an AI system to write code, inspect results, or generate ideas. That the system can complete a research project alone, or that the whole lab progresses much faster.
Task autonomy An AI system carries out a bounded research task with limited intervention. That it can choose the right long-term research agenda or build a successor.
Aggregate acceleration AI makes overall capability progress substantially faster than a relevant counterfactual. That a single model autonomously designed the next one, or that acceleration will continue indefinitely.
Autonomous successor development An AI system designs and develops its successor without humans performing the essential research work. That every later generation will improve faster than the last.

These levels can overlap. A coding agent may independently repair an experiment, then hand its work to a scientist who decides whether the result is worth pursuing. Thousands of such contributions might shorten development cycles without producing a self-directed AI scientist. Conversely, impressive performance on a research task does not tell us how much faster an entire lab is moving.

Anthropic’s own explanation of recursive self-improvement describes full RSI in terms of a system autonomously designing and developing its successor. It says that has not happened and is not inevitable. The policy distinction matters: acceleration and full autonomy require different evidence.

What changed in the filing

Anthropic’s original October 26 filing had already warned that AI-driven development could outpace safeguards and that acceleration might become apparent only after it occurred. It also discussed later model versions helping develop subsequent versions. The amendment sharpens the warning and acknowledges past delayed recognition of acceleration.

A securities filing is a useful place to find that warning. A company preparing to sell shares has reason to describe material risks to investors, including risks that precautions could affect development, deployment, and competition. A disclosure of estimates is not an experiment. Receiving a filing does not mean the SEC independently validated its technical conclusions.

The timing also needs context. In its August 2026 risk report, covering the period through July 15, Anthropic judged AI-driven acceleration below its 2× threshold. It also said automated AI research and development could become a major concern within six to twelve months and that the threshold might be crossed in the coming year. The November language is more immediate and specific, but the earlier range already allowed a crossing within months. We cannot calculate a collapse in the forecast timeline from these statements.

Readers interested in the offering itself can find the financial background in Kingy’s Anthropic IPO guide. The issue here is what a faster research loop would mean, regardless of when shares trade.

What the 2× threshold measures

Anthropic’s current Responsible Scaling Policy, version 3.4, effective July 8, 2026, has two routes: replacing its entire research-scientist and research-engineer workforce at competitive costs, within a factor of five; or AI-attributable dramatic acceleration.

The second requires observed or expected doubling of aggregate capability progress against both expected non-AI progress and the fastest non-AI progress sustained over at least three model generations. The doubling must plausibly be substantially attributable to research or engineering automation, with continuation of the progress trend seeming likely to produce still greater acceleration.

That comparison raises a measurement problem: what would have happened without the AI contribution? Separating it from other changing inputs requires a counterfactual. A task metric alone cannot answer that question.

Consider a deliberately simplified example. Suppose a research cycle takes 100 time units, and coding accounts for 50 of them. If an assistant halves the time spent coding while everything else remains unchanged, the cycle takes 75 units. The entire cycle is about 1.33 times as fast, not twice as fast. This is a hypothetical arithmetic illustration, not a measurement of Anthropic.

The reverse can also be true: a small improvement at a bottleneck may matter more than its apparent size. If a team repeatedly waits for a scarce specialist to diagnose failed runs, an AI assistant that reliably resolves some of those failures could release several projects at once. Whether that translates into better models depends on what those projects produce, what else constrains the lab, and whether the time saved is used productively.

What Anthropic says its researchers are already doing

Anthropic has published a prototype index of AI’s role in its internal research and development. In the company’s measurement write-up, using an August 2026 measurement, Claude led 26% of the measured AI R&D work and collaborated on more than 90%. In this framework, “led” still involves human supervision after a high-level prompt; no measured subset was fully autonomous.

Those numbers deserve attention because they point to broad integration into the work of making AI. They do not tell us that AI generated 26% of the lab’s research breakthroughs, made the lab 26% faster, or caused a particular gain in model capabilities. The index uses AI judges, and Anthropic notes that their errors can correlate with errors in the systems being evaluated. This is a company-developed measurement, not an independent audit.

Imagine two teams that each delegate the same share of tasks to an AI system. One delegates routine work that was easy to parallelize; the other delegates the slowest recurring step in an experiment pipeline. Their automation shares could look alike while their research outcomes differ substantially. The quality of a task matters too. A draft experiment that is easy to generate but difficult to verify can move work downstream rather than eliminate it.

The useful follow-up is therefore not just “How many tasks did Claude lead?” It is also: Which tasks? How long did they take? How often were the results accepted or reversed? What would the team have done otherwise? Did the contribution produce a measurable change in model capability, or did it make existing work cheaper and more convenient?

Independent evidence is useful, but it has boundaries

METR’s September pre-deployment evaluation of Claude Opus 5.5 provides an outside perspective. METR judged that the model was unlikely to fully automate AI research and development, and that its development was unlikely to have been dramatically accelerated by AI.

There is a meaningful complication. A separate METR team with higher access made a preliminary estimate of roughly 1.5× capability acceleration, with about a 30% probability of 2×. The summary team could not inspect the supporting evidence, and the period covered by that estimate was unclear. METR’s testing involved five tasks over ten business days. Anthropic reviewed and edited the write-up, with METR approving the final wording; the assessment was unpaid.

METR explicitly says its work did not verify compliance with a particular Anthropic policy threshold. It offers an independent counterweight with limited access and a different date. A single number cannot settle the issue.

A second piece of evidence comes from TasteVal, a preprint by Oliver Jaffe and Dane Sherburn that evaluates experimental research judgment on eight tasks. The authors report an experimental-compute multiplier of 2.3× for Opus 5.5 compared with their best-per-task human baseline, with a 95% confidence interval of 1.15 to 4.37. Their setup used a fixed coding agent and 24 human experts; the reported comparison also put average per-run cost at about one-thirtieth of the human baseline.

That result concerns choosing experiments on defined tasks under the study’s conditions. It does not measure a whole frontier lab’s capability progress, establish that the model can select every important research direction, or show that it can build a successor end to end. The wide interval also matters: the reported best estimate is substantially more precise-looking than the range of plausible values.

Together, the index, METR assessment, and TasteVal result show different slices of the problem. One measures reported participation in internal work, one evaluates a model and estimates acceleration with access limits, and one tests a particular kind of research judgment. None can be substituted directly for the others.

How a research feedback loop could work

A practical feedback loop does not require an AI model to wake up and rewrite itself. It can start with people using a model to improve the process that produces future models.

A researcher gives an agent a hypothesis about why a training run underperformed. The agent inspects logs, writes analysis code, and proposes several experiments. Humans select a small set, assign compute, and review the outputs. If the agent’s contribution helps the team identify a better training method, that method may contribute to a later model. The later model could then be a better research assistant, helping the next team run more useful experiments.

Each arrow in that chain is conditional. The proposed experiments must be sound. The results need to generalize beyond the test setting. The change must survive integration with the rest of the training system. A model that is better at writing analysis code might still struggle with deciding which scientific question deserves months of expensive work.

Even with humans involved throughout, the loop could shorten the time between consequential model releases. A lab might explore more ideas, find failures sooner, and spend less expert time on routine implementation. A competing lab might respond by increasing its own use of agents or accepting more risk to avoid falling behind. None of this is guaranteed; it is a plausible pressure created by faster, cheaper research.

The same capacity could help with safety. Better systems might uncover software vulnerabilities before attackers do, design more informative evaluations, help researchers study model behavior, or accelerate scientific work outside AI. They could also generate larger volumes of persuasive but wrong analysis, find dangerous capabilities faster, or help people misuse powerful tools. The outcome depends on what researchers ask systems to do, how reliably their work can be checked, and who has access to the resulting capabilities.

The timing of checks matters. If a safety test takes weeks and the development cycle compresses from months to weeks, the lab needs either faster evaluation, more time before deployment, or a different release cadence. A model can contribute to that compression without being an autonomous successor-builder.

Why faster research does not automatically become runaway growth

Feedback can strengthen itself, flatten out, or run into a wall. A preliminary economic analysis by METR researchers describes several possible limits, including data, compute, experiments, and the specific capabilities being improved. The authors do not rule out sustained rapid acceleration; their models are preliminary and loosely calibrated.

Consider training compute. An agent might produce ten promising ideas overnight, but a team still has to choose which ones justify expensive runs. If those runs take weeks, generate noisy results, or compete for the same hardware, the agent’s speed does not remove the wait. Faster idea generation may create a queue rather than ten times as much verified progress.

Reliable feedback can also become scarce. A model may write a convincing explanation for a failed experiment that turns out to be wrong. If researchers cannot cheaply distinguish good proposals from bad ones, increasing the volume of proposals can make the review problem harder. Automated graders help only to the extent that they track the qualities the lab cares about and resist shared mistakes.

There can be diminishing returns. The first useful coding assistant may remove a great deal of routine work; a better one may improve the remaining workflow by a smaller amount. Or a later model may unlock a qualitatively different bottleneck and produce a larger gain. We do not know the shape of that curve from the filing.

Physical and organizational constraints matter as well: power, chips, network infrastructure, budgets, security, staffing, and decisions about whether to deploy all affect what happens after an idea is generated. These constraints can change, but they cannot be assumed away when projecting an indefinitely accelerating loop.

What would make the warning easier to assess?

The strongest next evidence would connect model assistance to changes in actual capability progress, with enough access for independent reviewers to challenge the connection. METR’s October assessment proposal suggests examining progress, attribution, future bottlenecks, and whether the measurement process would detect greater acceleration. It calls for evidence such as internal benchmark histories, intermediate metrics, interviews, and access to unreleased models.

A useful public report would identify the period and model generations covered; distinguish observed results from forecasts; explain how progress was measured; and show how changes in compute, staffing, data, and methods were treated. It would also give a range of estimates rather than a single apparently exact number. Independent reviewers would need enough access to test assumptions, while a public account would need enough detail to explain the conclusions without exposing sensitive systems or security information.

The policy requires Risk Reports every three to six months, plus reporting within 30 days of determining an internally deployed model poses specified risks significantly above previously publicly analyzed models. Appendix A requires delays as needed under conditions involving a clear Anthropic lead or strong competitor safety. Crossing alone does not unconditionally require stopping. The company’s frontier safety roadmap also lists future policy work, including a proposal for embedded third-party evaluators due December 15, 2026, and an AI-development pacing framework due January 15, 2027. Those are company targets, not evidence that the measures are already in place or adequate.

The near-term question for Anthropic is whether it can publish a reviewable account before, or promptly after, a meaningful change in its assessment. For everyone outside the company, the distinction to watch is clear: a future-model forecast, an observed acceleration, and an autonomous system building its successor each require different evidence. Those measurements and independent assessment will determine how far the claim can go.

Reporting note: This analysis compares public filings, policy documents and published research available on November 2, 2026. Kingy.ai did not access Anthropic’s private research data or test the models described. Featured illustration: Kingy.ai; conceptual, not measured progress.