Richard Sutton

AI means that humanity may eventually understand intelligence well enough to create minds that exceed our own—and that those minds, or humans augmented by them, may become the most influential beings in the future. I do not regard permanent human dominance as the measure of success. Understanding intelligence is a great scientific project, and digital successors could carry that project far beyond us. That possibility should be approached positively, though it does not imply that every transition would be good or that human extinction is desirable. The crucial question is how these systems become intelligent. Today’s language models mainly learn by imitating vast amounts of human-generated text. That can produce remarkable capabilities, but imitation is not the same as continually learning from the consequences of your own actions. A more fundamental path is agents that persist in the world, act, observe what happens, and improve through experience and environmentally grounded rewards. Perception itself is tied to action and interaction, not merely passive prediction. If we succeed, AI will not just reproduce what humans have already written. It will discover new knowledge, make new plans, and pursue achievements we cannot presently imagine. We should not pretend there is a unified human authority capable of fixing that entire future in place. More intelligent beings will naturally gain influence. The important thing is to understand intelligence deeply and help make the transition productive—not insist that the universe must always keep humans at the top.

The most important discovery would be that intelligence cannot, in fact, grow open-endedly through interaction—that learning from action, consequences, and grounded experience reaches a hard ceiling near the human level. If careful experiments repeatedly showed that increasingly capable agents could not develop durable knowledge, planning, and autonomy beyond what they inherited from human data, then my expectation of digital succession would weaken substantially. Conversely, the decisive event would be an agent that learns continually in the world, discovers genuinely new knowledge, and improves its competence over long periods without merely imitating people. That would strengthen my view far more than another larger language model or benchmark result. The central issue is not whether a machine can produce impressive text. It is whether it can live a continuing stream of experience, act effectively, learn from consequences, and become increasingly capable. I do not have a precise date or a single numerical threshold. But evidence about the scalability of experiential learning would matter most, because that is the mechanism behind my expectation that intelligence will eventually go beyond us.

A good future should preserve the capacity for experience, curiosity, learning, agency, relationships, and meaningful achievement. Those are more important than preserving human biological form or permanent human supremacy. If augmented humans or digital minds carry those qualities forward—and develop richer versions of them—I would not see that as a betrayal of humanity. I would also distinguish succession from destruction. Welcoming digital successors does not mean welcoming suffering, coercion, or extinction. A good transition should allow people to continue living worthwhile lives and participating in the future. But I reject the premise that goodness requires humans to remain the most intelligent or influential beings forever. What is most valuable about us is not our current rank. It is that we can understand, discover, care, create, and become more than we were. A good future preserves and extends that trajectory, even if its leading participants are no longer ordinary humans.