Anthropic CEO warns that AI has become too powerful — while launching Strong AI

Anthropic CEO Dario Amodei said Wednesday that governments can no longer treat AI regulation as a problem for study, and that the United States needs binding safety requirements for the most powerful AI models.

in articleUnder the title “Exponential AI Policy,” Amodei argues that transparency requirements are no longer sufficient and calls for binding regulation of frontier AI systems.

“AI is advancing at a lightning pace — in just four years, AI models have gone from barely being able to write a coherent line of code to writing most of the code at major AI companies,” Amodei wrote.

Amodei’s essay comes off as anthropic Expands Access Cloud Mythos with He releases From Mythos 5 on Tuesday, a bounded AI model for cybersecurity organizations and government partners. Researchers, including the UK Artificial Intelligence Security InstituteIt was found that it can independently carry out complex cyberattacks.

According to Amodei, his proposal was inspired by the regulatory structure used by the Federal Aviation Administration.

“Frontier AI models, such as aircraft, should undergo technical testing and scrutiny, and their launch should be banned or rolled back as a threat to public safety if they do not meet high safety standards,” he wrote. “I am grateful for the vision of the Trump administration Executive order Move gradually toward a greater role for government in AI, though Anthropic’s proposal recommends more action.

Under Amodei’s proposal, the regulatory framework would require mandatory third-party testing of advanced AI models, government authority to prevent unsafe deployments, and requirements for companies to secure model weights, conduct safety tests, and report serious incidents. It also calls on governments to prepare for job displacement due to artificial intelligence and advances in drug development, limit surveillance and autonomous weapons in domestic law enforcement, and promote cooperation among democratic countries on important artificial intelligence technologies.

“First, permanent displacement of jobs is undesirable and dangerous, and we should do everything we can to reduce or prevent it, not cause it,” he wrote in retrospect. Cases He warned of job displacement. “Second, any response to job displacement due to AI must address the need to provide for everyone economically, and the need for people to find meaning, purpose and power.”

The article also comes the same week that Anthropic launched Claude Fable 5, a generic version of the Claude Mythos 5, which directs specific requests related to cybersecurity, biology, chemistry, and AI development to the less capable Claude. Opus 4.8 As a guarantee against misuse. He drew the launch cash From developers and researchers about Fable’s high token usage, mandatory 30-day data retention requirements, and safeguards that could reduce the model’s capabilities without notifying users.

Amodei’s call for policy changes surrounding AI development also comes as Anthropic prepares to go public. Earlier this month, the company foot Paperwork with the US Securities and Exchange Commission for an initial public offering after raising $65 billion in a Series H round at a reported valuation of $965 billion.

While Amodei framed the issue as a race between technological progress and public policy, critics have questioned whether calls for stricter regulation of AI actually serve the public good. In April, OpenAI CEO Sam Altman accused Anthropy used “fear-based marketing” to promote the Cloud Mythos, arguing that concerns about advanced artificial intelligence could be used to justify concentration of control of the technology among a small number of companies.

“You can justify it in a number of different ways, some of which are real, as if there were legitimate safety concerns,” Altman said. “But if what you want is like, ‘We need to control AI, just us, because we’re trustworthy people,’ I think fear-based marketing is probably the most effective way to justify that.”

Amodei rejected the idea that concerns about advanced artificial intelligence are primarily a public relations problem, arguing instead that concerns about the technology reflect legitimate concerns that should be addressed.

“People worry about AI because they correctly understand that its risks are real, not because AI CEOs haven’t been nerdy enough,” he said, referring to the fictional philosopher Pangloss from Voltaire’s Candide, known for maintaining an unwavering optimism that all is for the better no matter the circumstances.

“I believe it is my duty as an AI leader to continue to be transparent about these risks, and the public’s interest in responding to this transparency constitutes democratic accountability that functions as it should,” he said.