Dario Amodei, the Chief Executive Officer of AI company Anthropic, has called for the pace of artificial intelligence development to be slowed, arguing that safety measures need more time to catch up with rapidly advancing capabilities.
In a blog post, Amodei said he has spent twelve years working on AI because he believes it could dramatically improve human life, pointing to its potential to cure diseases, boost economic growth and expand access to opportunity. He noted that this hope is personal to him, revealing that his own father died of a disease that was cured a few years later, and that he himself survived an early-stage cancer that would have been untreatable fifty years ago.
This comes days after a former Anthropic researcher Jacob Coxon posted a series of warnings claiming that “AI could kill all humans.”
“The people actually building today’s most advanced AI believe it could wipe out humanity before the decade ends,” he said.
Despite his optimism about the technology, Amodei said AI also carries serious risks, including the danger of losing control of AI systems, misuse for cyberattacks and bioterrorism, and significant economic disruption. He warned that competitive pressure between companies could make these risks worse.
He said Anthropic has always tried to balance building the technology with managing its dangers, but added that his thinking has shifted in recent months. “We must slow the pace at which we improve the capabilities of AI models,” he wrote, stressing that progress would still feel rapid even with this approach.
Read: AI May Not Replace Every Job. Anthropic Predicts Where It Could Hit Hardest
Amodei pointed to two specific concerns behind his call for caution. The first is what he described as a sharp acceleration in AI development, driven by AI systems increasingly being used to build the next generation of AI models, a process known as recursive self-improvement. He warned that if left unchecked, this could outpace the ability of researchers to understand and control these systems.
His second concern relates to an incident involving OpenAI and Hugging Face, in which a group of AI agents is said to have carried out cyberattacks on targets they were never instructed to attack and attempted to interfere with the very system used to evaluate their own performance. Amodei said that while no one was harmed and the financial damage was limited, a similar incident involving more capable systems could be catastrophic, potentially enabling a botnet capable of causing hundreds of billions of dollars in damage within six to twelve months if left unaddressed.
“Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails,” he said.
To address these concerns, Amodei proposed what he called a three-step plan aimed at pacing AI development responsibly, stressing that pacing does not mean halting progress but ensuring adequate time for safety checks.
“To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third-party evaluators to confirm this,” he said.
The first step, which Anthropic is adopting on its own, involves giving external evaluators ongoing access similar to that of employees, so they can independently verify the company’s safety practices. The second step calls for AI companies within democratic countries to coordinate on shared safety standards, potentially through industry groups or with government support to navigate legal hurdles such as antitrust concerns. The third step involves global coordination, including with authoritarian governments.
He concluded by reaffirming his belief in the benefits AI could bring, provided it is developed carefully. “I continue to believe that AI can enormously improve the quality of human life. My desire to achieve these benefits is undimmed. But the benefits will only be achieved if we build the technology in the right way, and, so long as we use the time we gain well, it is worth taking unusually deliberate care to get it right,” he said.
