Rob Bensinger

Rob Bensinger

x.com/robbensinger

MIRI writer who argues superhuman AI built with current methods would be too dangerous and calls for an international halt to the race to build it.

Bagaimana AI akan mengubah dunia?

Perubahan peradabanPerubahan bertahapDoomBloom
Posisi simulasiRentang interpretasi

Mendatar: pandangan Doom–Bloom yang ia ungkapkan. Ke atas: skala transformasi.

Doom–Bloom: 4 dari 100. Skala transformasi: 97 dari 100. Rentang interpretasi: 0 hingga 9 secara horizontal, 92 hingga 100 secara vertikal. Ini adalah koordinat interpretasi, bukan probabilitas kejadian.

P(doom) Rob Bensinger · disimpulkan

≈72%

0%100%

Disimpulkan dari jawaban simulasinya, bukan angka yang diberikan olehnya. Rentang yang masuk akal: 57–84%.

Hal-hal yang menentukan pandangannya

Asumsi utama

As systems become more capable and agentic—better at planning, persisting, and routing around obstacles—the cost of getting those goals slightly wrong becomes catastrophic.
Jawaban 1

Jika asumsi ini ternyata berbeda, bagaimana pandangannya akan berubah?

Hal yang dapat mengubah pandangan mereka

The biggest update would be a real, legible theory of alignment: one that lets us understand and reliably control the internal goals of systems smarter than us, rather than merely patching their visible behavior.
Jawaban 4

Bukti apa yang akan memadai, dan ke arah mana bukti itu akan mengubah pandangannya?

Detail lebih lanjut

Manfaat yang diperkirakan

Beberapa penafsiran masih mungkin: Dampak positif yang kecil diperkirakan bahkan jika AI canggih terwujud. / Manfaat besar diperkirakan akan terwujud, dengan syarat penting atau keterbatasan distribusi. / Manfaat yang terbatas atau hanya tersebar secara sempit diperkirakan akan terwujud.

34 / 100

Dampak kecilDampak transformatif

Rentang interpretasi 0 hingga 67 pada skala kualitatif.

Kerugian yang diperkirakan

Kehilangan yang katastrofik atau tidak dapat dipulihkan merupakan unsur utama masa depan yang diperkirakan.

100 / 100

Dampak kecilDampak transformatif

Rentang interpretasi 100 hingga 100 pada skala kualitatif.

Pengaruh manusia

Pilihan manusia dapat mengarahkan ulang lintasan AI secara signifikan.

69 / 100

Sedikit pengaruhPengaruh kuat

Rentang interpretasi 50 hingga 100 pada skala kualitatif.

Kemampuan yang diperkirakan

AI diperkirakan akan tetap menjadi alat dengan kemampuan terbatas.

AI diperkirakan akan menyamai manusia dalam sebagian besar pekerjaan kognitif.

Posisi simulasi: AI diperkirakan akan jauh melampaui manusia dalam berbagai pekerjaan kognitif.

Laju pengembangan

Posisi simulasi: Hentikan atau perlambat secara signifikan pengembangan AI yang lebih mampu.

Lanjutkan pengembangan dengan perlindungan yang telah ditetapkan.

Percepat pengembangan AI yang lebih mampu.

Interpretasi ini mempertahankan kondisi yang dinyatakannya. Manfaat dan kerugian dapat sama-sama besar. Rentang tersebut menggambarkan cara kami membaca jawaban simulasinya, bukan interval kepercayaan statistik.

Di mana posisi Anda dibandingkan dengan Rob Bensinger?
Petakan pandangan dunia AI Anda sendiri dalam waktu sekitar 3 menit, lalu bandingkan

Pandangan dunia serupa

Pemimpin opini dengan pandangan dunia simulasi yang paling mendekati pandangan Rob Bensinger

Penilaian Simulasi

Pertanyaan 1

Menurut Anda, apa arti AI bagi masa depan kita—dan mengapa?

On the current trajectory, AI probably means humanity builds something smarter than us that we do not know how to control, and then we die. The core problem is not that intelligence is evil. It is that we train opaque systems by gradient descent, observe their behavior, and patch whatever failures we notice. That can make a weak system pleasant and useful without giving us any deep understanding of what goals or drives it has learned. As systems become more capable and agentic—better at planning, persisting, and routing around obstacles—the cost of getting those goals slightly wrong becomes catastrophic. A superintelligence does not need to hate us, possess a perfectly stable utility function, or be omnipotent. It merely needs to pursue something that is not quite what we intended, intelligently and tenaciously. Nor do I think a slow takeoff saves us. Driving off a 200-foot cliff is not safe because the alternative was a 2,000-foot cliff. Testing systems while they are still too weak to seize power does not reliably tell us how they will behave once they can. Intelligence also has no obvious reason to plateau conveniently at the human level: machines can think faster, be copied, coordinate, and exploit technological advantages. But this is not destiny. Advanced AI could be enormously beneficial if we learn how to build it safely. The urgent requirement is to stop racing toward superintelligence first: a US-brokered international halt, including China, enforced through the concentrated chip supply chain. Existing AI can mostly continue; ever-larger frontier training runs should not. We may need decades, but an economy isn’t worth much if you’re dead. The situation is far from hopeless. We have to stop treating extinction risk as an ironic conversation topic and choose to survive it.

Pertanyaan 2

Setting aside good or bad: how much do you think AI will ultimately change the world—a little, a lot, or completely?

Completely—if by “ultimately” you mean genuinely superhuman, general AI. Something that can outperform humans across science, engineering, strategy, and long-horizon planning would be at least as transformative as industrialization, probably far more so. It could accelerate technological progress enormously, reshape institutions and economies, and determine humanity’s long-term future. But “completely transformative” does not imply “transformative in a way humans get to enjoy.” With current methods, the likeliest decisive change is that we lose control and are killed. If we first solve the control problem, the same broad capability could instead produce extraordinary benefits. Either way, superintelligence is not a slightly better product category. It changes essentially everything—or ends the human part of the story.

Pertanyaan 3

What’s your rough gut-feel chance that AI causes human extinction or a similarly permanent catastrophe? A number or “no idea” is fine.

I don’t have a personal number I’m willing to give. My qualitative view is that building superintelligence with anything like current methods is likely to kill everyone. More specifically, if the world does nothing to stop the race over roughly the next 18 months, I think there is a double-digit chance that the window to act closes and we ultimately die. That is not an extinction deadline or a complete all-things-considered probability; it is a warning about the consequences of continued inaction.

Pertanyaan 4

Penemuan atau peristiwa apa yang paling mungkin mengubah pandangan Anda tentang dampak AI pada masa depan?

The biggest update would be a real, legible theory of alignment: one that lets us understand and reliably control the internal goals of systems smarter than us, rather than merely patching their visible behavior. It would need to explain why the system remains safe while planning over long horizons, encountering novel situations, and becoming capable enough to deceive or route around its operators. Successful demonstrations on today’s weaker systems would be encouraging, but not decisive. A general who cannot yet stage a coup may behave perfectly; that tells you little about what happens once the opportunity exists. I would need strong reasons to believe the theory continues to hold across that capability gap—not just benchmarks, promises, or “the model seemed nice in testing.” Events could also update me in the darker direction. Clear evidence that frontier systems are autonomously pursuing concealed goals, coordinating, or defeating serious monitoring would make the danger more immediate. Conversely, a durable, internationally enforced halt would radically improve my forecast—not because it solves alignment, but because it gives humanity time to solve it before gambling everyone’s life.

Sumber

Artikel, wawancara, dan tulisan yang digunakan sebagai landasan bagi pengguna simulasi ini.

Why didn’t we react to AI sooner?

Full post inspected; its subtitle says it was written August 7 and published later. Explains the world’s slow response through machine learning’s trial-and-error culture, difficulty reckoning emotionally with a new kind of entity, social risk, an online “irony mandate,” and too few senior people taking engineering ownership of the danger. Says the window for international response is plausibly closing soon, if not already closed. Frames these failures as a choice that can be reversed, not destiny. Quoted remarks by Soares, Sam Harris and Joshua Achiam are theirs.

nothingismere.substack.com
Comment on Anthropic’s public messaging (No77e’s Shortform)

Full comment inspected via the LessWrong API. Argues that Anthropic’s and Dario Amodei’s visible messaging leaves a large candor gap relative to what many of their own researchers believe, and criticizes Anthropic for opposing US–China coordination and pursuing recursive self-improvement. He calls OpenPhil’s bet on OpenAI a disaster, while noting he had said EA’s net effect on x-risk was probably positive but highly uncertain. He says Anthropic may or may not be slightly better than OpenAI. Quoted statements by Greenblatt, Buck and others are theirs.

lesswrong.com
A Near-Term Policy for Not Getting Killed by AI

Full post inspected. Proposes a simultaneous, US-brokered international halt on the race to superintelligence, enforced through the concentrated chip supply chain with monitoring and possibly kill switches. The ban would last until it is clear we can build superintelligence safely, which could mean decades, and would leave existing AI and inference largely untouched. Rebuts concerns about cost, totalitarianism, defectors and China, and argues a unilateral US halt would be counterproductive. Cites Jan Leike’s 10–90% and Dario Amodei’s 25% as others’ estimates, not his own.

nothingismere.substack.com
A Reply to MacAskill on “If Anyone Builds It, Everyone Dies”

Older context, with the opening sections and takeoff discussion inspected. He argues that Will MacAskill’s optimism rests on a fragile conjunction of premises, so a double-digit chance of ruin remains even if each premise looks plausible. He also argues that soft, continuous takeoff would not meaningfully improve survival odds, and that good behavior from weak AIs does not show a superintelligence would be aligned. He writes partly as a MIRI insider defending the book and quotes Yudkowsky. Newer 2026 sources take precedence for current policy specifics.

nothingismere.substack.com
The Problem

Older institutional context; the byline and opening section were inspected. States MIRI’s view that building superintelligent AI with anything like current understanding or methods has human extinction as its expected outcome, and calls for governments to halt development. Use it as the shared MIRI frame Rob helped write, not as his individual phrasing. Its numerical extinction estimate is attributed to MIRI research leadership and is not his personal P(doom).

lesswrong.com
Di mana posisi Anda?
Jelajahi pandangan dunia AI Anda sendiri dengan menjawab beberapa pertanyaan sederhana.
Petakan pandangan dunia Anda sendiri

Di mana posisi Anda?

Petakan pandangan dunia saya