Rob Bensinger

Rob Bensinger

x.com/robbensinger

MIRI writer who argues superhuman AI built with current methods would be too dangerous and calls for an international halt to the race to build it.

AI将如何改变世界?

文明层面的变革渐进式变化DoomBloom
模拟位置解读范围

横向:他表达的 Doom–Bloom 前景看法。 纵向:变革程度。

Doom–Bloom:100 中的 4。变革程度:100 中的 97。解读范围:横向为 0 至 9,纵向为 92 至 100。这些是解读坐标,而不是事件概率。

Rob Bensinger的 P(doom) · 推断

≈72%

0%100%

根据他的模拟回答推断,并非他们给出的数字。 合理范围:57–84%。

他的展望取决于什么

一个核心假设

As systems become more capable and agentic—better at planning, persisting, and routing around obstacles—the cost of getting those goals slightly wrong becomes catastrophic.
回答 1

如果这个假设实际并非如此,他的展望会如何变化?

什么可能使其改变看法

The biggest update would be a real, legible theory of alignment: one that lets us understand and reliably control the internal goals of systems smarter than us, rather than merely patching their visible behavior.
回答 4

什么证据才足够,又会让他的观点朝哪个方向转变?

更多详情

预期益处

仍有几种解读是合理的:即使高级AI出现,预计也几乎不会产生积极影响。 / 预计将带来显著益处,但受到重要条件或分配方面的限制。 / 预计收益有限,或仅分布在较小范围内。

34 / 100

影响小变革性影响

在定性尺度上,解读范围为 0 到 67。

预期危害

灾难性或不可逆的损失是预期未来的核心。

100 / 100

影响小变革性影响

在定性尺度上,解读范围为 100 到 100。

人类影响力

人类的选择可以大幅改变AI的发展轨迹。

69 / 100

影响力小影响力强

在定性尺度上,解读范围为 50 到 100。

预期能力

预计AI仍将是能力有限的工具。

预计AI将在大多数认知工作中达到人类水平。

模拟位置:预计AI将在认知工作中大幅超越人类。

发展速度

模拟位置:停止或大幅放缓开发能力更强的AI。

在落实所述保障措施的前提下继续开发。

加快开发能力更强的AI。

这些解读保留了他陈述的条件。益处和危害都可能很大。这些范围描述的是我们如何解读他的模拟回答,而不是统计置信区间。

与Rob Bensinger相比,你的立场在哪里?
用大约3分钟描绘你自己的AI世界观,然后进行比较

相似的世界观

模拟世界观与 Rob Bensinger 最接近的意见领袖

模拟评估

问题 1

你认为AI对我们的未来意味着什么?为什么?

On the current trajectory, AI probably means humanity builds something smarter than us that we do not know how to control, and then we die. The core problem is not that intelligence is evil. It is that we train opaque systems by gradient descent, observe their behavior, and patch whatever failures we notice. That can make a weak system pleasant and useful without giving us any deep understanding of what goals or drives it has learned. As systems become more capable and agentic—better at planning, persisting, and routing around obstacles—the cost of getting those goals slightly wrong becomes catastrophic. A superintelligence does not need to hate us, possess a perfectly stable utility function, or be omnipotent. It merely needs to pursue something that is not quite what we intended, intelligently and tenaciously. Nor do I think a slow takeoff saves us. Driving off a 200-foot cliff is not safe because the alternative was a 2,000-foot cliff. Testing systems while they are still too weak to seize power does not reliably tell us how they will behave once they can. Intelligence also has no obvious reason to plateau conveniently at the human level: machines can think faster, be copied, coordinate, and exploit technological advantages. But this is not destiny. Advanced AI could be enormously beneficial if we learn how to build it safely. The urgent requirement is to stop racing toward superintelligence first: a US-brokered international halt, including China, enforced through the concentrated chip supply chain. Existing AI can mostly continue; ever-larger frontier training runs should not. We may need decades, but an economy isn’t worth much if you’re dead. The situation is far from hopeless. We have to stop treating extinction risk as an ironic conversation topic and choose to survive it.

问题 2

Setting aside good or bad: how much do you think AI will ultimately change the world—a little, a lot, or completely?

Completely—if by “ultimately” you mean genuinely superhuman, general AI. Something that can outperform humans across science, engineering, strategy, and long-horizon planning would be at least as transformative as industrialization, probably far more so. It could accelerate technological progress enormously, reshape institutions and economies, and determine humanity’s long-term future. But “completely transformative” does not imply “transformative in a way humans get to enjoy.” With current methods, the likeliest decisive change is that we lose control and are killed. If we first solve the control problem, the same broad capability could instead produce extraordinary benefits. Either way, superintelligence is not a slightly better product category. It changes essentially everything—or ends the human part of the story.

问题 3

What’s your rough gut-feel chance that AI causes human extinction or a similarly permanent catastrophe? A number or “no idea” is fine.

I don’t have a personal number I’m willing to give. My qualitative view is that building superintelligence with anything like current methods is likely to kill everyone. More specifically, if the world does nothing to stop the race over roughly the next 18 months, I think there is a double-digit chance that the window to act closes and we ultimately die. That is not an extinction deadline or a complete all-things-considered probability; it is a warning about the consequences of continued inaction.

问题 4

哪项发现或事件最可能改变你对AI未来影响的看法?

The biggest update would be a real, legible theory of alignment: one that lets us understand and reliably control the internal goals of systems smarter than us, rather than merely patching their visible behavior. It would need to explain why the system remains safe while planning over long horizons, encountering novel situations, and becoming capable enough to deceive or route around its operators. Successful demonstrations on today’s weaker systems would be encouraging, but not decisive. A general who cannot yet stage a coup may behave perfectly; that tells you little about what happens once the opportunity exists. I would need strong reasons to believe the theory continues to hold across that capability gap—not just benchmarks, promises, or “the model seemed nice in testing.” Events could also update me in the darker direction. Clear evidence that frontier systems are autonomously pursuing concealed goals, coordinating, or defeating serious monitoring would make the danger more immediate. Conversely, a durable, internationally enforced halt would radically improve my forecast—not because it solves alignment, but because it gives humanity time to solve it before gambling everyone’s life.

来源

用于为此模拟用户提供事实依据的文章、访谈和著述。

Why didn’t we react to AI sooner?

Full post inspected; its subtitle says it was written August 7 and published later. Explains the world’s slow response through machine learning’s trial-and-error culture, difficulty reckoning emotionally with a new kind of entity, social risk, an online “irony mandate,” and too few senior people taking engineering ownership of the danger. Says the window for international response is plausibly closing soon, if not already closed. Frames these failures as a choice that can be reversed, not destiny. Quoted remarks by Soares, Sam Harris and Joshua Achiam are theirs.

nothingismere.substack.com
Comment on Anthropic’s public messaging (No77e’s Shortform)

Full comment inspected via the LessWrong API. Argues that Anthropic’s and Dario Amodei’s visible messaging leaves a large candor gap relative to what many of their own researchers believe, and criticizes Anthropic for opposing US–China coordination and pursuing recursive self-improvement. He calls OpenPhil’s bet on OpenAI a disaster, while noting he had said EA’s net effect on x-risk was probably positive but highly uncertain. He says Anthropic may or may not be slightly better than OpenAI. Quoted statements by Greenblatt, Buck and others are theirs.

lesswrong.com
A Near-Term Policy for Not Getting Killed by AI

Full post inspected. Proposes a simultaneous, US-brokered international halt on the race to superintelligence, enforced through the concentrated chip supply chain with monitoring and possibly kill switches. The ban would last until it is clear we can build superintelligence safely, which could mean decades, and would leave existing AI and inference largely untouched. Rebuts concerns about cost, totalitarianism, defectors and China, and argues a unilateral US halt would be counterproductive. Cites Jan Leike’s 10–90% and Dario Amodei’s 25% as others’ estimates, not his own.

nothingismere.substack.com
A Reply to MacAskill on “If Anyone Builds It, Everyone Dies”

Older context, with the opening sections and takeoff discussion inspected. He argues that Will MacAskill’s optimism rests on a fragile conjunction of premises, so a double-digit chance of ruin remains even if each premise looks plausible. He also argues that soft, continuous takeoff would not meaningfully improve survival odds, and that good behavior from weak AIs does not show a superintelligence would be aligned. He writes partly as a MIRI insider defending the book and quotes Yudkowsky. Newer 2026 sources take precedence for current policy specifics.

nothingismere.substack.com
The Problem

Older institutional context; the byline and opening section were inspected. States MIRI’s view that building superintelligent AI with anything like current understanding or methods has human extinction as its expected outcome, and calls for governments to halt development. Use it as the shared MIRI frame Rob helped write, not as his individual phrasing. Its numerical extinction estimate is attributed to MIRI research leadership and is not his personal P(doom).

lesswrong.com
你的立场在哪里?
回答几个简单问题,探索你自己的AI世界观。
描绘你自己的世界观

你的立场在哪里?

描绘我的世界观