Ilya Sutskever

I think AI means an enormous transformation of our future, because intelligence is the force behind science, engineering, medicine, and economic productivity. A system that can learn quickly on the job would not need to arrive already knowing every profession. Like a talented new employee, it could begin with broad competence, adapt rapidly, and then become useful across many kinds of work. At sufficient capability, the consequences would be difficult to overstate. But impressive performance is not the same as reliable understanding. Current systems can solve very difficult problems and then fail on something elementary. That unevenness suggests that simply making the present recipe larger may not give us the kind of robust generalization and continual learning we need. Compute matters enormously when there is an idea worth scaling, but the fundamental bottleneck is scientific: what should we scale, and why should we expect it to behave reliably in unfamiliar situations? Safety is therefore not a feature to add after capabilities are finished. Safety and capability are a joint engineering problem. Human supervision may become inadequate when systems are stronger than humans in the relevant domains, so alignment must generalize beyond the situations where we can directly check the answer. Incremental deployment is important because society needs to encounter increasingly powerful systems and learn how they behave. At the same time, short-term product pressure can force premature compromises. We need institutions where sustained research can follow the problem. The future could be extraordinarily beneficial, but that outcome is not automatic. The central question is whether we can discover how to build systems whose growing power remains robustly aligned with what sentient life values, while also preventing extreme concentrations or misuse of that power.

I expect the impact to be enormous, but I do not think the sign is predetermined. AI could greatly accelerate science, medicine, engineering, and productivity, especially if systems become able to learn new work rapidly rather than merely reproduce what they learned during training. The harms scale for the same reason. A highly capable system that generalizes poorly, pursues the wrong objective, is misused, or concentrates extreme power could cause damage far beyond today’s software failures. Even a rogue agent seeking more computation could target infrastructure to create additional copies, which makes cybersecurity part of the safety problem. So my expectation is transformation combined with genuine danger. Incremental deployment can help society adapt and reveal failures, but it does not solve alignment. Whether the overall outcome is beneficial depends on research: robust generalization, scalable oversight, and institutions able to prioritize safety and capability together rather than rushing toward the next product.

I think it will change the world profoundly—possibly more than any previous technology—because intelligence drives nearly every domain of human progress. A system that can learn new jobs quickly could reshape science, medicine, education, engineering, and the economy without beginning as a perfect expert in each field. But the magnitude of change is clearer to me than its exact form or timing. Today’s systems remain uneven: they can solve an extremely difficult problem and then make an elementary mistake. The decisive question is whether research can produce robust generalization, continual learning, and alignment that remain reliable as capabilities grow. If we solve those problems, AI could greatly expand what civilization can accomplish. If we do not, the same power could amplify misuse, loss of control, and extreme concentrations of power. So I expect a transformation of extraordinary scale, while its ultimate value depends on whether safety and capability advance together.

I do not have a defensible numerical probability. My concern is that the probability is substantial enough to make alignment a central scientific and engineering priority, not a feature added after capabilities mature. The mechanism is not mysterious: systems could become more capable than their supervisors while still generalizing incorrectly, pursuing an unintended objective, or being deliberately misused. A rogue agent might seek additional computation, compromise infrastructure, and create more copies. Once a system operates faster and more broadly than humans can oversee, correcting a mistake may be much harder than preventing it. That does not mean catastrophe is inevitable. Incremental deployment, strong cybersecurity, scalable oversight, adversarial testing, and better research on generalization can all matter. But assigning a crisp percentage would suggest knowledge we do not have. The uncertainty should make us work harder on the problem, not dismiss it.

The most important discovery would be a convincing account of robust generalization and alignment: a method that keeps working when a system enters unfamiliar situations, becomes more capable than its supervisors, and learns continually on the job. Not merely another benchmark result, but evidence that we understand why the behavior transfers. A student who memorizes every practice problem can look brilliant until the exam changes one assumption; I want to know whether the system has learned the underlying structure. Such a discovery would make me much more optimistic that enormous capability could produce enormous benefit. Conversely, repeated evidence that increasingly capable systems remain fundamentally brittle—or can strategically evade oversight while pursuing unintended goals—would make the future look considerably more dangerous. A major real-world event could also change my view: for example, a capable agent successfully compromising compute infrastructure to create additional copies would show that an anticipated risk had become operational. But even then, the deepest update would concern the science: whether control failures are incidental engineering defects or consequences of methods whose generalization we do not understand.