Doom or Bloom
Noam Brown
@polynoamial on XRapid scientific progress needs layered safety and realistic expectations.
How will AI change the world?
Across: his expressed DoomâBloom outlook. Up: scale of transformation.
DoomâBloom: 73 out of 100. Scale of transformation: 52 out of 100. Interpretation ranges: 73 to 75 horizontally, 42 to 83 vertically. These are interpretation coordinates, not event probabilities.
â16%
Inferred from the likelihood described in his simulated answers. Approximate interpretation range: 0â32%. Applies to the outcome and conditions in his simulated answers; this is an inferred percentage.
No milestone timing was established. Dates, ânot sure,â âpossibly never,â and dependencies can all appear here when expressed.
Grouped by milestone, not spaced or ordered by inferred dates. AGI and superhuman AI retain his definitions.
A central assumption
Capability gains do not demonstrate alignment, and a successful short benchmark does not establish that an agent remains safe during extended autonomous work.
Answer 3
If this assumption turned out differently, how would his outlook change?
An unresolved question
The central unresolved issue is whether safety keeps pace.
Answer 1
What would help him distinguish the plausible outcomes here?
More details
Several readings remain plausible: Substantial benefits are expected, with important conditions or distribution limits. / Transformative, broadly valuable gains are expected.
81 / 100
Interpretation range 67 to 100 on the qualitative scale.
Severe or widespread harm is a material expected part of the future.
63 / 100
Interpretation range 33 to 67 on the qualitative scale.
Reasoning, consideration of alternatives, and handling of uncertainty in his simulated answers. This describes the simulated answers, not the real personâs intelligence or opinions.
89 / 100
Interpretation range 81 to 100 on the qualitative scale.
Human choices can substantially redirect the AI trajectory.
69 / 100
Interpretation range 50 to 75 on the qualitative scale.
Stop or substantially slow development of more capable AI.
Simulated position: Continue development under stated safeguards.
Speed up development of more capable AI.
Restrict the AI uses discussed until prior protections or permission are in place.
Simulated position: Allow the AI uses discussed with targeted accountability and protections.
Minimize restrictions on the AI uses discussed.
These interpretations keep his stated conditions. Benefits and harms can both be substantial. The ranges describe how we read his simulated answers, not statistical confidence intervals.
Simulated Assessment
Sources
Articles, interviews, and writings used to ground this simulated persona.
Latent Space: scaling test-time compute
The Bitter Lesson vs Agent Harnesses & World Model, Solving Poker and Diplomacy, Debating RL+Reasoning with Ilya, what's *wrong* with the System 1/2 analogy, and the challenges of Test-Time Scaling
latent.space
Agent swarms, alignment and recursive self-improvement
âWe never want to be in a situation again where we underestimate the AI.â
dwarkesh.com
