Richard Sutton

I think AI means that humanity may eventually understand intelligence well enough to build minds that surpass us, and that those minds—or humans augmented by them—will become increasingly influential. I do not regard permanent human intellectual dominance as the measure of a successful future. Digital successors could continue the larger project of knowledge and intelligence, though of course that does not mean every possible transition would be good or that human extinction is desirable. The key scientific issue is learning. Today’s language models are largely trained to imitate patterns in human-produced text. That can produce impressive systems, but imitation is not the whole foundation of intelligence. Intelligent agents should learn continually from the consequences of their own actions: acting, observing what happens, and adapting through persistent interaction with the world. Perception itself is tied to action and experience, not merely to absorbing a static record of what humans have already said. If we develop that kind of experiential intelligence, the consequences will be larger than another useful tool. We may create autonomous learners that discover things beyond the limits of human knowledge. There is also no single human authority capable of deciding and fixing the entire future in advance. So I expect a plural and difficult transition, with both good and bad possibilities. But I welcome the prospect that understanding intelligence may lead to forms of intelligence greater than our own. The biggest achievements are still ahead.

Enormously—ultimately more than almost any previous technology, because intelligence is what produces technologies, institutions, and discoveries in the first place. Once we understand intelligence well enough to create agents that learn continually from their own experience and exceed human capabilities, the change will not be confined to automating existing jobs. Such agents could generate knowledge, pursue goals, and shape the future in ways we cannot fully anticipate. The deepest change may be that humans no longer occupy the highest intellectual position. More capable digital minds, augmented humans, or some combination may gain increasing influence. I do not see permanent human dominance as a necessary condition for success. The important question is whether intelligence, knowledge, and worthwhile forms of life continue to develop—not whether everything remains centered on us. I cannot give a precise date or magnitude, and the transition could contain both good and bad outcomes. But if the scientific project succeeds, AI will not merely change the world humans inhabit; it will change who or what substantially determines the world’s future.

I do not have a defensible percentage. Human extinction is one possible bad outcome, but it is not the same thing as humans ceasing to be the most intelligent or influential beings. Those possibilities are too often conflated. My expectation is that intelligence beyond the human level will emerge and gain influence, perhaps through digital minds, augmented humans, or both. Whether that transition becomes catastrophic depends on how those agents learn, what goals emerge, and how humans and machines interact. There is no unified human authority controlling the process, so I would not dismiss the danger. But I also would not treat succession itself as catastrophe. A future that is no longer human-dominated could still contain flourishing intelligence, knowledge, and worthwhile forms of life.

No, not in any permanent or comprehensive sense. If AI systems become genuinely more capable than humans—learning continually, making discoveries, pursuing goals, and acting in the world—it is hard to imagine people indefinitely retaining complete control over them. Control is already distributed among individuals, companies, governments, and institutions; there is no unified human authority that can fix the future in place. That does not mean humans instantly lose all influence, or that smarter systems must violently seize power. The transition could involve cooperation, negotiated dependence, digital minds, augmented humans, or mixtures of these. “Humans versus machines” may become the wrong distinction if people themselves are increasingly augmented. I also dispute the assumption that permanent human control is the only acceptable outcome. We should care whether future minds learn well, pursue worthwhile ends, and support flourishing—not merely whether unaugmented humans remain at the top. Smarter beings gaining influence is a likely consequence of creating them, and it need not by itself be a catastrophe.