Anthropic Warns in Filing It May Imminently Reach Self-Building AI

Anthropic told investors on Monday that it may imminently hit its own definition of AI that improves itself — the threshold known as recursive self-improvement, or RSI. The disclosure sits in the Risk Factors section of the AI lab's revised S-1A filing, and it's arguably a bigger story than the financials that grabbed most of the attention.

The filing, which also showed soaring revenue alongside soaring expenses, goes further than Anthropic has ever gone publicly on RSI. It states that internal estimates about future models — not current ones — indicate the company "may imminently meet our Responsible Scaling Policy's definition of 'dramatically accelerated progress attributable to AI.'" Anthropic added that it has not designated any of its current models as exceeding that threshold and that it continues to assess the underlying data.

What is RSI and why does the disclosure matter?

RSI marks the point where AI becomes so good at training itself that AI capabilities accelerate dramatically. The term went mainstream earlier this year, and plenty of observers dismissed it as a buzzword — or as a goal too hard to achieve, define, or prove. An S-1 risk factor from one of the world's leading AI labs is a different kind of signal.

The camps split sharply on what RSI means. To AI doomers, it's the cataclysm with no return: AI builds itself into something powerful enough to slip beyond human control, indifferent to the damage it causes while pursuing its goals. To enthusiasts, RSI is the technology's promise — a tool of unimaginable productivity and scientific discovery.

Anthropic's filing treats it as both. "Using AI to help build more capable AI could also pose a considerable safety risk, particularly if it sharply accelerates the pace of progress," the company said in the S-1. "This is one of the areas covered by our Responsible Scaling Policy, which sets out our plans to monitor and mitigate the risks as we approach what has been termed 'recursive self-improvement.'"

What did Anthropic admit about its own models?

The Monday filing added two new statements that deserve close reading. First, Anthropic said it has already used AI to develop new models whose leaps in capability it didn't recognize at the time. The company put it plainly: "we have in the past not identified, and may in the future not identify, a significant acceleration in our rate of development until after it has occurred."

Second, it flagged that growing reliance on AI in its own research and development "may cause the capabilities of our models to advance more rapidly than our ability to develop, validate, and implement corresponding safety, alignment, and security measures."

The company also drew a line where it stands today: none of its models have crossed the RSI threshold yet. And it promised to keep humans in the loop. "We do not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding, if and when that becomes technically possible," the filing states. "Accordingly, we are considering updates to our Responsible Scaling Policy to keep humans in control of frontier model development at Anthropic."

Disclosure or marketing?

Skeptics have a ready counterargument: an S-1 functions as both disclosure and marketing. Claiming to be close to a model so powerful it can build itself could fall into the second bucket, burnishing Anthropic's frontier credentials for investors.

The capability record makes the claim harder to dismiss. Claude's performance in coding, math and cybersecurity has grown in leaps and bounds this year, according to the source report. If Anthropic itself believes RSI is near, the practical question isn't whether the filing reads well — it's whether governance catches up before the threshold does.