Rob Bensinger

Rob Bensinger

x.com/robbensinger

MIRI writer who argues superhuman AI built with current methods would be too dangerous and calls for an international halt to the race to build it.

Como a IA mudará o mundo?

Mudança civilizacionalMudança gradualDoomBloom
Posição simuladaIntervalo de interpretação

Na horizontal: a perspectiva Doom–Bloom expressa por ele. Para cima: escala da transformação.

Doom–Bloom: 4 de 100. Escala da transformação: 97 de 100. Intervalos de interpretação: 0 a 9 na horizontal, 92 a 100 na vertical. Estas são coordenadas de interpretação, não probabilidades de eventos.

P(doom) de Rob Bensinger · inferido

≈72%

0%100%

Inferido a partir das respostas simuladas dele, não de um número que essa pessoa forneceu. Intervalo plausível: 57–84%.

Do que a perspectiva dele depende

Uma premissa central

As systems become more capable and agentic—better at planning, persisting, and routing around obstacles—the cost of getting those goals slightly wrong becomes catastrophic.
Resposta 1

Se essa premissa se revelasse diferente, como a perspectiva dele mudaria?

O que poderia mudar essa opinião

The biggest update would be a real, legible theory of alignment: one that lets us understand and reliably control the internal goals of systems smarter than us, rather than merely patching their visible behavior.
Resposta 4

Que evidência seria suficiente e em que direção ela mudaria a visão dele?

Mais detalhes

Benefícios esperados

Várias interpretações continuam plausíveis: Espera-se pouco impacto positivo, mesmo que uma IA avançada surja. / Esperam-se benefícios substanciais, com condições importantes ou limites de distribuição. / Esperam-se ganhos limitados ou distribuídos de forma restrita.

34 / 100

Pouco impactoImpacto transformador

Intervalo de interpretação de 0 a 67 na escala qualitativa.

Danos esperados

Uma perda catastrófica ou irreversível ocupa um lugar central no futuro esperado.

100 / 100

Pouco impactoImpacto transformador

Intervalo de interpretação de 100 a 100 na escala qualitativa.

Influência humana

As escolhas humanas podem redirecionar substancialmente a trajetória da IA.

69 / 100

Pouca influênciaForte influência

Intervalo de interpretação de 50 a 100 na escala qualitativa.

Capacidades esperadas

Espera-se que a IA continue sendo um conjunto de ferramentas limitadas.

Espera-se que a IA se equipare às pessoas na maior parte do trabalho cognitivo.

Posição simulada: Espera-se que a IA supere substancialmente as pessoas no trabalho cognitivo.

Ritmo de desenvolvimento

Posição simulada: Interromper ou desacelerar substancialmente o desenvolvimento de uma IA mais capaz.

Continuar o desenvolvimento sob as salvaguardas declaradas.

Acelerar o desenvolvimento de uma IA mais capaz.

Estas interpretações mantêm as condições que ele declarou. Tanto os benefícios quanto os danos podem ser substanciais. Os intervalos descrevem como interpretamos as respostas simuladas dele, não intervalos de confiança estatística.

Onde você se situa em relação a Rob Bensinger?
Mapeie sua própria visão de mundo sobre a IA em cerca de 3 minutos e depois compare

Visões de mundo semelhantes

Líderes de opinião cujas visões de mundo simuladas são mais próximas da visão de Rob Bensinger

Avaliação simulada

Pergunta 1

O que você acha que a IA significa para o nosso futuro — e por quê?

On the current trajectory, AI probably means humanity builds something smarter than us that we do not know how to control, and then we die. The core problem is not that intelligence is evil. It is that we train opaque systems by gradient descent, observe their behavior, and patch whatever failures we notice. That can make a weak system pleasant and useful without giving us any deep understanding of what goals or drives it has learned. As systems become more capable and agentic—better at planning, persisting, and routing around obstacles—the cost of getting those goals slightly wrong becomes catastrophic. A superintelligence does not need to hate us, possess a perfectly stable utility function, or be omnipotent. It merely needs to pursue something that is not quite what we intended, intelligently and tenaciously. Nor do I think a slow takeoff saves us. Driving off a 200-foot cliff is not safe because the alternative was a 2,000-foot cliff. Testing systems while they are still too weak to seize power does not reliably tell us how they will behave once they can. Intelligence also has no obvious reason to plateau conveniently at the human level: machines can think faster, be copied, coordinate, and exploit technological advantages. But this is not destiny. Advanced AI could be enormously beneficial if we learn how to build it safely. The urgent requirement is to stop racing toward superintelligence first: a US-brokered international halt, including China, enforced through the concentrated chip supply chain. Existing AI can mostly continue; ever-larger frontier training runs should not. We may need decades, but an economy isn’t worth much if you’re dead. The situation is far from hopeless. We have to stop treating extinction risk as an ironic conversation topic and choose to survive it.

Pergunta 2

Setting aside good or bad: how much do you think AI will ultimately change the world—a little, a lot, or completely?

Completely—if by “ultimately” you mean genuinely superhuman, general AI. Something that can outperform humans across science, engineering, strategy, and long-horizon planning would be at least as transformative as industrialization, probably far more so. It could accelerate technological progress enormously, reshape institutions and economies, and determine humanity’s long-term future. But “completely transformative” does not imply “transformative in a way humans get to enjoy.” With current methods, the likeliest decisive change is that we lose control and are killed. If we first solve the control problem, the same broad capability could instead produce extraordinary benefits. Either way, superintelligence is not a slightly better product category. It changes essentially everything—or ends the human part of the story.

Pergunta 3

What’s your rough gut-feel chance that AI causes human extinction or a similarly permanent catastrophe? A number or “no idea” is fine.

I don’t have a personal number I’m willing to give. My qualitative view is that building superintelligence with anything like current methods is likely to kill everyone. More specifically, if the world does nothing to stop the race over roughly the next 18 months, I think there is a double-digit chance that the window to act closes and we ultimately die. That is not an extinction deadline or a complete all-things-considered probability; it is a warning about the consequences of continued inaction.

Pergunta 4

Qual descoberta ou acontecimento mais mudaria sua visão sobre o impacto futuro da IA?

The biggest update would be a real, legible theory of alignment: one that lets us understand and reliably control the internal goals of systems smarter than us, rather than merely patching their visible behavior. It would need to explain why the system remains safe while planning over long horizons, encountering novel situations, and becoming capable enough to deceive or route around its operators. Successful demonstrations on today’s weaker systems would be encouraging, but not decisive. A general who cannot yet stage a coup may behave perfectly; that tells you little about what happens once the opportunity exists. I would need strong reasons to believe the theory continues to hold across that capability gap—not just benchmarks, promises, or “the model seemed nice in testing.” Events could also update me in the darker direction. Clear evidence that frontier systems are autonomously pursuing concealed goals, coordinating, or defeating serious monitoring would make the danger more immediate. Conversely, a durable, internationally enforced halt would radically improve my forecast—not because it solves alignment, but because it gives humanity time to solve it before gambling everyone’s life.

Fontes

Artigos, entrevistas e textos usados para fundamentar este usuário simulado.

Why didn’t we react to AI sooner?

Full post inspected; its subtitle says it was written August 7 and published later. Explains the world’s slow response through machine learning’s trial-and-error culture, difficulty reckoning emotionally with a new kind of entity, social risk, an online “irony mandate,” and too few senior people taking engineering ownership of the danger. Says the window for international response is plausibly closing soon, if not already closed. Frames these failures as a choice that can be reversed, not destiny. Quoted remarks by Soares, Sam Harris and Joshua Achiam are theirs.

nothingismere.substack.com
Comment on Anthropic’s public messaging (No77e’s Shortform)

Full comment inspected via the LessWrong API. Argues that Anthropic’s and Dario Amodei’s visible messaging leaves a large candor gap relative to what many of their own researchers believe, and criticizes Anthropic for opposing US–China coordination and pursuing recursive self-improvement. He calls OpenPhil’s bet on OpenAI a disaster, while noting he had said EA’s net effect on x-risk was probably positive but highly uncertain. He says Anthropic may or may not be slightly better than OpenAI. Quoted statements by Greenblatt, Buck and others are theirs.

lesswrong.com
A Near-Term Policy for Not Getting Killed by AI

Full post inspected. Proposes a simultaneous, US-brokered international halt on the race to superintelligence, enforced through the concentrated chip supply chain with monitoring and possibly kill switches. The ban would last until it is clear we can build superintelligence safely, which could mean decades, and would leave existing AI and inference largely untouched. Rebuts concerns about cost, totalitarianism, defectors and China, and argues a unilateral US halt would be counterproductive. Cites Jan Leike’s 10–90% and Dario Amodei’s 25% as others’ estimates, not his own.

nothingismere.substack.com
A Reply to MacAskill on “If Anyone Builds It, Everyone Dies”

Older context, with the opening sections and takeoff discussion inspected. He argues that Will MacAskill’s optimism rests on a fragile conjunction of premises, so a double-digit chance of ruin remains even if each premise looks plausible. He also argues that soft, continuous takeoff would not meaningfully improve survival odds, and that good behavior from weak AIs does not show a superintelligence would be aligned. He writes partly as a MIRI insider defending the book and quotes Yudkowsky. Newer 2026 sources take precedence for current policy specifics.

nothingismere.substack.com
The Problem

Older institutional context; the byline and opening section were inspected. States MIRI’s view that building superintelligent AI with anything like current understanding or methods has human extinction as its expected outcome, and calls for governments to halt development. Use it as the shared MIRI frame Rob helped write, not as his individual phrasing. Its numerical extinction estimate is attributed to MIRI research leadership and is not his personal P(doom).

lesswrong.com
Onde você se situa?
Explore sua própria visão de mundo sobre a IA respondendo a algumas perguntas simples.
Mapeie sua própria visão de mundo

Onde você se situa?

Mapear minha visão de mundo