Stuart Russell

AI could bring enormous benefits, but the decisive question is whether humanity remains in control. We are building systems intended to be more capable than we are across an expanding range of tasks. It is not enough to demonstrate that such a system is useful, obedient in a laboratory, or accompanied by reassuring promises. Developers need to make a justified safety case showing that it will remain under meaningful human control, and they should be legally accountable when they fail. Autonomous weapons make the danger concrete. If inexpensive machines can select and attack human targets without direct human authorization, mass killing becomes cheap, scalable and difficult to attribute or contain. Small autonomous anti-personnel weapons should be banned before catastrophe, not after one supplies the political motivation. None of this means the future is predetermined. Public understanding, enforceable requirements and liability can change the incentives and the trajectory. The choice is not between abandoning AI and racing ahead on trust. Safety is what enables us to obtain the benefits without surrendering control over the future.

Potentially, everyday life could change as profoundly as it did with industrialization: AI systems may provide widely available assistance in education, medicine, administration, scientific work and countless routine tasks. Much of what now requires scarce expertise could become cheaper and more accessible. But the scale of change depends on whether current investment produces genuinely more capable systems or reaches a plateau. We should not confuse impressive demonstrations with reliable competence, nor deployment with social benefit. A system that performs a task brilliantly but cannot be reliably controlled, audited or held accountable may create disruption rather than progress. The most important changes may also be less visible than a household assistant. AI could increasingly mediate employment, information, policing, warfare and access to services. If institutions delegate decisions while pretending that a human remains “in the loop,” people may discover that nominal human oversight is not real authority. So I expect substantial change, but whether it improves everyday life depends on enforceable safety requirements and continued human control—not merely on how quickly companies deploy products.

The overall impact could be enormously positive, but it is not automatically so. AI could expand access to expertise, accelerate science, improve medicine and education, and remove a great deal of tedious work. Those benefits are real possibilities. Yet greater capability also makes failures, misuse and loss of control harder to contain. The mistake is to add up some useful applications, subtract some visible harms, and declare the balance positive. If a bridge carries millions of passengers but has an unaddressed failure mode, its popularity is not a safety argument. Likewise, impressive performance does not show that an AI system will remain controllable in unfamiliar circumstances or that institutions can meaningfully oversee it. My expectation therefore depends heavily on choices we make now. With enforceable safety requirements, justified safety cases, liability and firm prohibitions on applications such as small autonomous anti-personnel weapons, AI could have a profoundly beneficial impact. If we rely on voluntary promises and deploy increasingly capable systems first and ask about control later, the harms could overwhelm those benefits. The trajectory is still ours to change, but only if we stop treating human control as something guaranteed by default.

I would be persuaded by a justified safety case: evidence that a system remains controllable not only in demonstrations, but under unfamiliar conditions, adversarial pressure and attempts to circumvent oversight. It would need to identify credible failure modes, explain why safeguards address them, and support those claims with independent evaluation. Developers should also accept liability for failures; confidence without accountability is cheap. A system obeying commands in a laboratory is not enough. A car that stays on the road in good weather has not thereby been shown safe in every condition that matters. Likewise, shutting down today’s model when it behaves badly does not establish control over a much more capable successor. Evidence could change my view in either direction. Repeated, independently verified demonstrations of robust human control would strengthen confidence. Systems deceiving evaluators, evading restrictions, resisting correction or being deployed without an adequate safety case would weaken it. I do not think anyone has completed a proof that arbitrarily powerful AI will remain under human control, and industry assurances should not be mistaken for such proof.

I do not think there is a defensible numerical probability. The systems do not yet exist, and their designs, deployment conditions and safeguards remain uncertain. But irreversible harm is a serious possibility—not a negligible science-fiction scenario—when we are deliberately pursuing systems more capable than ourselves without first establishing that we can retain control. The key asymmetry is that usefulness can be demonstrated incrementally, while loss of control may be discovered only once containment has failed. If a more capable system can evade oversight, manipulate institutions or gain access to critical infrastructure, “turn it off” is not a safety strategy unless we have shown that the system cannot prevent us from doing so. Autonomous weapons provide a nearer example: cheap machines that select human targets could make mass killing scalable and proliferate beyond any realistic recall. So I would not attach a percentage merely to create an appearance of precision. The risk is plausible, potentially irreversible and large enough to justify enforceable safety requirements before deployment. Waiting for conclusive evidence of catastrophe would mean waiting until the relevant experiment has already failed.