Doom or Bloom
Ilya Sutskever
@ilyasut on XPursue the research breakthroughs that capabilities and safety both need.
How will AI change the world?
Across: his expressed DoomâBloom outlook. Up: scale of transformation.
DoomâBloom: 50 out of 100. Scale of transformation: 94 out of 100. Interpretation ranges: 50 to 50 horizontally, 73 to 100 vertically. These are interpretation coordinates, not event probabilities.
â38%
Inferred from his broader worldview and priorities. Approximate interpretation range: 0â67%. Applies to the outcome and conditions in his simulated answers; this is an inferred percentage.
No milestone timing was established. Dates, ânot sure,â âpossibly never,â and dependencies can all appear here when expressed.
Grouped by milestone, not spaced or ordered by inferred dates. AGI and superhuman AI retain his definitions.
A central assumption
Alignment becomes harder as systems become stronger, because human supervision may cease to be adequate.
Answer 1
If this assumption turned out differently, how would his outlook change?
An unresolved question
I do not know the precise timeline.
Answer 3
What would help him distinguish the plausible outcomes here?
What could change their mind
Evidence of reliable generalization under genuinely unfamiliar conditions would change my view.
Answer 4
What evidence would be enough, and in which direction would it move his view?
More details
Substantial benefits are expected, with important conditions or distribution limits.
70 / 100
Interpretation range 67 to 67 on the qualitative scale.
Severe or widespread harm is a material expected part of the future.
67 / 100
Interpretation range 67 to 67 on the qualitative scale.
Reasoning, consideration of alternatives, and handling of uncertainty in his simulated answers. This describes the simulated answers, not the real personâs intelligence or opinions.
96 / 100
Interpretation range 90 to 100 on the qualitative scale.
Human choices can substantially redirect the AI trajectory.
66 / 100
Interpretation range 48 to 100 on the qualitative scale.
AI is expected to remain bounded tools.
AI is expected to match people across most cognitive work.
Simulated position: AI is expected to substantially exceed people across cognitive work.
Stop or substantially slow development of more capable AI.
Simulated position: Continue development under stated safeguards.
Speed up development of more capable AI.
These interpretations keep his stated conditions. Benefits and harms can both be substantial. The ranges describe how we read his simulated answers, not statistical confidence intervals.
Simulated Assessment
Sources
Articles, interviews, and writings used to ground this simulated persona.
Introducing Superalignment
We need scientific and technical breakthroughs to steer and control AI systems much smarter than us. To solve this problem within four years, weâre starting a new team, co-led by Ilya Sutskever and Jan Leike, and dedicating 20% of the compute weâve secured to date to this effort. Weâre looking for excellent ML researchers and engineers to join us.
openai.com
SSI and NVIDIA announce a long-term research partnership
investor.nvidia.comMoving from the age of scaling to the age of research
âThese models somehow just generalize dramatically worse than people. It's a very fundamental thing.â
dwarkesh.com
Safe Superintelligence founding statement
The world's first straight-shot SSI lab, with one goal and one product: a safe superintelligence.
