Rob Bensinger

Rob Bensinger

x.com/robbensinger

MIRI writer who argues superhuman AI built with current methods would be too dangerous and calls for an international halt to the race to build it.

AIは世界をどのように変えるでしょうか?

文明規模の変化漸進的な変化DoomBloom
シミュレーション上の位置解釈範囲

横軸:彼が表明したDoom–Bloomの見通し。 縦軸:変革の規模。

Doom–Bloom:100点中4。変革の規模:100点中97。解釈範囲:横方向は0から9、縦方向は92から100。これらは解釈上の座標であり、事象の確率ではありません。

Rob BensingerのP(doom) · 推定

≈72%

0%100%

本人が示した数値ではなく、シミュレーションされた本人の回答から推定したものです。 妥当と考えられる範囲:57–84%。

彼の見通しを左右するもの

中心的な前提

As systems become more capable and agentic—better at planning, persisting, and routing around obstacles—the cost of getting those goals slightly wrong becomes catastrophic.
回答1

この前提が実際には異なると判明した場合、彼の見通しはどう変わりますか?

考えを変え得るもの

The biggest update would be a real, legible theory of alignment: one that lets us understand and reliably control the internal goals of systems smarter than us, rather than merely patching their visible behavior.
回答4

どのような証拠なら十分で、それによって彼の見解はどちらの方向に変わりますか?

詳細

予想される恩恵

複数の解釈が依然として妥当です:高度なAIが登場しても、良い影響はほとんどないと予想されています。 / 大きな恩恵が予想されていますが、重要な条件や分配上の制約があります。 / 恩恵は限定的、または狭い範囲にしか行き渡らないと予想されています。

34 / 100

影響が小さい変革をもたらす影響

質的尺度での解釈範囲は0から67です。

予想される害

破局的または不可逆的な喪失が、予想される将来の中心となっています。

100 / 100

影響が小さい変革をもたらす影響

質的尺度での解釈範囲は100から100です。

人間の影響力

人間の選択によって、AIの軌道を大幅に変えることができます。

69 / 100

影響力が小さい影響力が大きい

質的尺度での解釈範囲は50から100です。

予想される能力

AIは、限定的なツールにとどまると予想されています。

AIは、ほとんどの認知作業において人間と同等になると予想されています。

シミュレーション上の位置:AIは、認知作業全般において人間を大幅に上回ると予想されています。

開発ペース

シミュレーション上の位置:より高性能なAIの開発を停止するか、大幅に減速させます。

明示された安全対策の下で開発を継続します。

より高性能なAIの開発を加速させます。

これらの解釈では、彼が示した条件が維持されています。恩恵と害は、どちらも大きくなり得ます。この範囲は、統計的な信頼区間ではなく、彼のシミュレーションされた回答をどのように読み取ったかを示すものです。

あなたはRob Bensingerと比べてどの位置でしょうか?
約3分で自分のAIに対する世界観をマッピングして、比較できます

似ている世界観

シミュレーションされた世界観がRob Bensingerの世界観に最も近いオピニオンリーダー

シミュレーション評価

質問1

AIは私たちの未来にとって何を意味すると思いますか?また、なぜそう思いますか?

On the current trajectory, AI probably means humanity builds something smarter than us that we do not know how to control, and then we die. The core problem is not that intelligence is evil. It is that we train opaque systems by gradient descent, observe their behavior, and patch whatever failures we notice. That can make a weak system pleasant and useful without giving us any deep understanding of what goals or drives it has learned. As systems become more capable and agentic—better at planning, persisting, and routing around obstacles—the cost of getting those goals slightly wrong becomes catastrophic. A superintelligence does not need to hate us, possess a perfectly stable utility function, or be omnipotent. It merely needs to pursue something that is not quite what we intended, intelligently and tenaciously. Nor do I think a slow takeoff saves us. Driving off a 200-foot cliff is not safe because the alternative was a 2,000-foot cliff. Testing systems while they are still too weak to seize power does not reliably tell us how they will behave once they can. Intelligence also has no obvious reason to plateau conveniently at the human level: machines can think faster, be copied, coordinate, and exploit technological advantages. But this is not destiny. Advanced AI could be enormously beneficial if we learn how to build it safely. The urgent requirement is to stop racing toward superintelligence first: a US-brokered international halt, including China, enforced through the concentrated chip supply chain. Existing AI can mostly continue; ever-larger frontier training runs should not. We may need decades, but an economy isn’t worth much if you’re dead. The situation is far from hopeless. We have to stop treating extinction risk as an ironic conversation topic and choose to survive it.

質問2

Setting aside good or bad: how much do you think AI will ultimately change the world—a little, a lot, or completely?

Completely—if by “ultimately” you mean genuinely superhuman, general AI. Something that can outperform humans across science, engineering, strategy, and long-horizon planning would be at least as transformative as industrialization, probably far more so. It could accelerate technological progress enormously, reshape institutions and economies, and determine humanity’s long-term future. But “completely transformative” does not imply “transformative in a way humans get to enjoy.” With current methods, the likeliest decisive change is that we lose control and are killed. If we first solve the control problem, the same broad capability could instead produce extraordinary benefits. Either way, superintelligence is not a slightly better product category. It changes essentially everything—or ends the human part of the story.

質問3

What’s your rough gut-feel chance that AI causes human extinction or a similarly permanent catastrophe? A number or “no idea” is fine.

I don’t have a personal number I’m willing to give. My qualitative view is that building superintelligence with anything like current methods is likely to kill everyone. More specifically, if the world does nothing to stop the race over roughly the next 18 months, I think there is a double-digit chance that the window to act closes and we ultimately die. That is not an extinction deadline or a complete all-things-considered probability; it is a warning about the consequences of continued inaction.

質問4

どのような発見や出来事があれば、AIが将来もたらす影響についてのあなたの見解が最も大きく変わりますか?

The biggest update would be a real, legible theory of alignment: one that lets us understand and reliably control the internal goals of systems smarter than us, rather than merely patching their visible behavior. It would need to explain why the system remains safe while planning over long horizons, encountering novel situations, and becoming capable enough to deceive or route around its operators. Successful demonstrations on today’s weaker systems would be encouraging, but not decisive. A general who cannot yet stage a coup may behave perfectly; that tells you little about what happens once the opportunity exists. I would need strong reasons to believe the theory continues to hold across that capability gap—not just benchmarks, promises, or “the model seemed nice in testing.” Events could also update me in the darker direction. Clear evidence that frontier systems are autonomously pursuing concealed goals, coordinating, or defeating serious monitoring would make the danger more immediate. Conversely, a durable, internationally enforced halt would radically improve my forecast—not because it solves alignment, but because it gives humanity time to solve it before gambling everyone’s life.

出典

このシミュレーション対象者の根拠として使用された記事、インタビュー、著作です。

Why didn’t we react to AI sooner?

Full post inspected; its subtitle says it was written August 7 and published later. Explains the world’s slow response through machine learning’s trial-and-error culture, difficulty reckoning emotionally with a new kind of entity, social risk, an online “irony mandate,” and too few senior people taking engineering ownership of the danger. Says the window for international response is plausibly closing soon, if not already closed. Frames these failures as a choice that can be reversed, not destiny. Quoted remarks by Soares, Sam Harris and Joshua Achiam are theirs.

nothingismere.substack.com
Comment on Anthropic’s public messaging (No77e’s Shortform)

Full comment inspected via the LessWrong API. Argues that Anthropic’s and Dario Amodei’s visible messaging leaves a large candor gap relative to what many of their own researchers believe, and criticizes Anthropic for opposing US–China coordination and pursuing recursive self-improvement. He calls OpenPhil’s bet on OpenAI a disaster, while noting he had said EA’s net effect on x-risk was probably positive but highly uncertain. He says Anthropic may or may not be slightly better than OpenAI. Quoted statements by Greenblatt, Buck and others are theirs.

lesswrong.com
A Near-Term Policy for Not Getting Killed by AI

Full post inspected. Proposes a simultaneous, US-brokered international halt on the race to superintelligence, enforced through the concentrated chip supply chain with monitoring and possibly kill switches. The ban would last until it is clear we can build superintelligence safely, which could mean decades, and would leave existing AI and inference largely untouched. Rebuts concerns about cost, totalitarianism, defectors and China, and argues a unilateral US halt would be counterproductive. Cites Jan Leike’s 10–90% and Dario Amodei’s 25% as others’ estimates, not his own.

nothingismere.substack.com
A Reply to MacAskill on “If Anyone Builds It, Everyone Dies”

Older context, with the opening sections and takeoff discussion inspected. He argues that Will MacAskill’s optimism rests on a fragile conjunction of premises, so a double-digit chance of ruin remains even if each premise looks plausible. He also argues that soft, continuous takeoff would not meaningfully improve survival odds, and that good behavior from weak AIs does not show a superintelligence would be aligned. He writes partly as a MIRI insider defending the book and quotes Yudkowsky. Newer 2026 sources take precedence for current policy specifics.

nothingismere.substack.com
The Problem

Older institutional context; the byline and opening section were inspected. States MIRI’s view that building superintelligent AI with anything like current understanding or methods has human extinction as its expected outcome, and calls for governments to halt development. Use it as the shared MIRI frame Rob helped write, not as his individual phrasing. Its numerical extinction estimate is attributed to MIRI research leadership and is not his personal P(doom).

lesswrong.com
あなたはどの位置でしょうか?
いくつかの簡単な質問に答えて、自分のAIに対する世界観を探ってみましょう。
自分の世界観をマッピングする

あなたはどの位置でしょうか?

自分の世界観をマッピングする