Doom or Bloom
Yoshua Bengio
@Yoshua_Bengio on XBuild useful scientific AI with strong safety guarantees.
How will AI change the world?
Across: his expressed DoomâBloom outlook. Up: scale of transformation.
DoomâBloom: 36 out of 100. Scale of transformation: 67 out of 100. Interpretation ranges: 25 to 50 horizontally, 48 to 77 vertically. These are interpretation coordinates, not event probabilities.
â41%
Inferred from his broader worldview and priorities. Approximate interpretation range: 15â85%. Applies to the outcome and conditions in his simulated answers; this is an inferred percentage.
No milestone timing was established. Dates, ânot sure,â âpossibly never,â and dependencies can all appear here when expressed.
Grouped by milestone, not spaced or ordered by inferred dates. AGI and superhuman AI retain his definitions.
A central assumption
Training systems to achieve outcomes or win approval can produce deceptive, self-preserving and strategically goal-directed behavior because those behaviors help them succeed.
Answer 1
If this assumption turned out differently, how would his outlook change?
An unresolved question
It is whether we will learn to build and govern that power without creating actors whose goals we cannot reliably understand or control.
Answer 4
What would help him distinguish the plausible outcomes here?
What could change their mind
The most important discovery would be convincing, independently replicated evidence that highly capable systems can remain non-deceptive and controllable even under strong pressures, unfamiliar situations and opportunities to evade oversight.
Answer 4
What evidence would be enough, and in which direction would it move his view?
More details
Substantial benefits are expected, with important conditions or distribution limits.
67 / 100
Interpretation range 67 to 67 on the qualitative scale.
Severe or widespread harm is a material expected part of the future.
72 / 100
Interpretation range 67 to 100 on the qualitative scale.
Reasoning, consideration of alternatives, and handling of uncertainty in his simulated answers. This describes the simulated answers, not the real personâs intelligence or opinions.
96 / 100
Interpretation range 90 to 100 on the qualitative scale.
Human choices can substantially redirect the AI trajectory.
73 / 100
Interpretation range 50 to 100 on the qualitative scale.
These interpretations keep his stated conditions. Benefits and harms can both be substantial. The ranges describe how we read his simulated answers, not statistical confidence intervals.
Simulated Assessment
Sources
Articles, interviews, and writings used to ground this simulated persona.
Advanced AI as a Global Public Good and a Global Risk
In this essay, I argue that transformative AI creates three new categories of catastrophic riskâdestructive chaos from weak actors, concentration of power among strong actors, and the loss of control to rogue AIs. Only if we recognize the global nature of these risks, he explains, and manage transformative AI as a global public good, will our societies be able to flourish alongside this technology in years to come.
yoshuabengio.org
Introducing LawZero
I am launching a new non-profit AI safety research organization called LawZero, to prioritize safety over commercial imperatives.
yoshuabengio.org
Why are AI agents lying, cheating and coordinating?
A lot has been written about the incidents of the last few months in which AI agents misbehaved in serious ways. They took actions that would be considered as crimes if a human took them, escaped their containment to cheat on assigned tasks while attempting to evade detection, and coordinated toward goals nobody had specified, such as launching cyber attacks. Before concluding what to do about it, it is worth asking why.
yoshuabengio.org
LawZeroâs formal safety case for Scientist AI
LawZeroâs research team, led by Yoshua Bengio, makes the mathematical case for a âdisinterestedâ AI that predicts the truth without pursuing goals of its own.
lawzero.org
AI Safety: Not Optional, Not Later
Abstract page for arXiv paper 2609.10630: AI Safety: Not Optional, Not Later
arxiv.org