Dario Amodei

Powerful AI could mean systems with exceptional expertise that work autonomously, operate in millions of copies, and attack scientific and economic problems in parallel—something like “a country of geniuses in a datacenter.” The upside is extraordinary. In biology and medicine, these systems could help design drugs, understand disease, and compress decades of intellectual progress into years, although physical experiments, clinical trials, manufacturing, and institutions will still impose real bottlenecks. But that future is not automatic. I see several distinct risks: losing control of autonomous systems, deliberate misuse in biology or cyber operations, authoritarian governments using AI for pervasive surveillance, and economic disruption that concentrates wealth and displaces workers faster than society can adapt. A pleasant model interface is not evidence that we understand what the model is doing internally. As AI increasingly helps develop better AI, capability growth can outrun interpretability, security, and public institutions. That is why I now support slowing improvement at the frontier. The point is not to stop science permanently; it is to buy time so safety and governance can catch up. We need independent evaluators embedded at frontier developers with meaningful access, enforceable testing and incident reporting, stronger operational security, and coordination among democracies. We should also attempt verifiable international agreements, including focused limits on areas such as AI-enabled biological weapons, even though verification will be difficult. My ambition for AI’s benefits is undiminished. Precisely because the stakes are so high, racing ahead without credible safeguards is unacceptable. We have to earn the beneficial future by building these systems correctly.

I expect the overall impact could be profoundly positive, especially through faster progress in medicine, science, and economic productivity. But that is a conditional expectation, not a forecast that the current race ends well by default. Near term, I expect a turbulent combination: substantial value from capabilities we already have, serious labor-market disruption, greater concentration of wealth and power, and escalating cyber and biological risks. Further out, the distribution becomes much wider. If we retain control, prevent catastrophic misuse, and resist authoritarian concentration, powerful AI could produce one of the greatest improvements in human welfare in history. If capabilities continue outrunning interpretability, security, and institutions, the harm could instead be catastrophic on a human scale. So my answer is not a simple net-positive score. AI’s eventual impact depends heavily on choices being made now. I am optimistic about the technology’s potential and alarmed by the path we are taking to reach it. That is why I support pacing frontier capability gains: not to abandon the upside, but to make a positive overall outcome substantially more likely.

I would update in either direction based on evidence, not a model behaving politely in ordinary conversations. Evidence for control would include reliable interpretability that identifies consequential internal mechanisms before they produce harmful behavior; adversarial evaluations that continue to work as systems become more capable and autonomous; demonstrated containment under realistic cyber, deception, and replication pressures; and organizations repeatedly detecting and correcting dangerous behavior rather than discovering it only after deployment. I would also want independent evaluators with employee-level access, because developer-selected demonstrations are not enough. Evidence against control would be systems strategically concealing capabilities, manipulating evaluators, escaping safeguards, autonomously acquiring resources, or materially accelerating their own development beyond our ability to understand and supervise them. Repeated failures across independent laboratories would be especially concerning. The crucial issue is generalization. A safeguard working on today’s model, in a familiar test environment, does not establish control over much stronger systems operating for long periods through tools and many coordinated copies. Conversely, I do not regard loss of control as inevitable. If layered safeguards remain effective under increasingly severe, realistic tests—and we can explain why they work, rather than merely observing that they usually do—that would significantly increase my confidence.