Anthropic calls for verifiable pause mechanism for frontier AI development

Anthropic said more than 80% of the code merged into its codebase in May was written by Claude, a striking milestone that sharpened its warning that frontier AI may soon need a verifiable pause mechanism before the technology outruns human oversight.

In a report published June 4 titled When AI builds itself, the San Francisco company argued that recursive self-improvement is no longer a distant theory. Anthropic said systems that can help build their own successors would make securing, monitoring and shaping them far harder, because the pace of capability gains could move beyond the ability of institutions to respond.

The company’s answer is not a unilateral slowdown by one lab. Anthropic said any meaningful pause would require multiple well-resourced frontier labs to agree in advance on when development should slow or stop, what conditions would trigger the pause, who would decide when to lift it and how compliance would be overseen. Without those rules, Anthropic warned, a single company stepping back could simply cede ground to less cautious rivals.

That is the core credibility problem in the safety debate. A pause that cannot be independently verified would amount to little more than a promise, and promises are weak currency in a race defined by commercial pressure and national advantage. Anthropic said coordination would likely have to extend across major companies in multiple countries, especially the United States and China, if the agreement were to matter before a crisis hit.

The company’s own pace of development shows why it believes the issue is urgent. Anthropic said engineers now ship about 8 times as much code per quarter as they did from 2021 to 2025, and that the average task length AI systems can handle on their own has been doubling roughly every four months, down from an earlier seven-month doubling rate. On that trajectory, Anthropic said AI may reach tasks that take days this year and tasks that take weeks in 2027.

Anthropic also said it planned to convene policymakers, researchers, civil society groups and other AI firms in the coming months to discuss recursive self-improvement and coordination mechanisms. The pitch comes as the company has become one of the most influential voices urging restraint in an industry still rewarded for speed, even as it recently completed a fundraising round valuing it at $965 billion and confidentially filed for a U.S. initial public offering. The tension is clear: Anthropic is asking rivals to agree to a pause that would only work if everyone believed everyone else would actually stop.