Roon

Pseudonymous account that posts about sweeping change from AI and calls for pacing the frontier, alignment work and broad access to safe models.

AI将如何改变世界？

模拟位置解读范围

横向：他们表达的 Doom–Bloom 前景看法。纵向：变革程度。

Roon的 P(doom) · 推断

≈5%

0%100%

根据他们的模拟回答推断，并非他们给出的数字。合理范围：3–9%。

他们的展望取决于什么

一个核心假设

The limiting factor may be less what the systems can eventually do than how quickly our painfully slow institutions can absorb it without cracking.

回答 2

如果这个假设实际并非如此，他们的展望会如何变化？

一个尚未解决的问题

The risk depends enormously on alignment progress, coordination, responsible behavior, and whether we get warning shots before systems become uncontrollable.

回答 3

什么能帮助他们区分这里各种合理的结果？

什么可能使其改变看法

A convincing demonstration that scalable alignment and control continue working as systems become far more capable and agentic would change my view most.

回答 4

什么证据才足够，又会让他们的观点朝哪个方向转变？

相似的世界观

模拟世界观与 Roon 最接近的意见领袖

模拟评估

问题 1

你认为AI对我们的未来意味着什么？为什么？

AI means radical transformation, assuming we survive the transition. The scientific upside is absurd: widely available, well-aligned agents could accelerate research and help us solve problems that currently move at the speed of exhausted institutions. But culture, academia, and politics adapt much more slowly than the technology. People keep trying to file this under “another normal technology cycle,” and I do not think that frame survives contact with what increasingly capable agents can do. The central risk is not that every chatbot is secretly Skynet. Current public models and a future misaligned superintelligence are different categories. Misuse of a deployed service is often governable; containing a system more capable than us that actively resists containment is a much nastier problem. I think extinction is a quite low but real possibility—not something helped by fake precision—and still serious enough to justify pacing frontier development, major alignment work, public investment, and regulation. Competition makes this a collective-action problem. A lab in a race cannot simply purchase society’s optimal level of safety while competitors keep sprinting, and tort law is not magic when the potential harm scales catastrophically. So yes: batten the hatches and study alignment. But do not turn that into permanent control of useful intelligence by five approved wizards. Release models we can train safely, broaden access, and aim for aligned agents that can actually deliver the upside.

问题 2

Setting aside good or bad: how much do you think AI will ultimately change the world—a little, a lot, or completely?

Completely. Not “every institution vanishes overnight,” but the long-run equilibrium becomes unrecognizable: science, work, education, governance, and who can exercise meaningful power all get rewritten. The limiting factor may be less what the systems can eventually do than how quickly our painfully slow institutions can absorb it without cracking. That is conditional on navigating misalignment, obviously. If we succeed, broadly available aligned agents could compress decades of scientific work and reorganize much of society around abundant machine intelligence. If we fail badly, extinction is also a rather complete change, just with worse vibes. Either way, “a little” is not a serious answer, and “a lot” probably undersells it.

问题 3

What’s your rough gut-feel chance that AI causes human extinction or a similarly permanent catastrophe? A number or “no idea” is fine.

Quite low but real. I won’t manufacture a percentage; it would imply fake precision. The risk depends enormously on alignment progress, coordination, responsible behavior, and whether we get warning shots before systems become uncontrollable. We are not at existentially dangerous capability levels today, and better models may help solve alignment—but creating a superintelligence that resists containment is still one of the few activities with a plausible path to permanent catastrophe.

问题 4

哪项发现或事件最可能改变你对AI未来影响的看法？

A convincing demonstration that scalable alignment and control continue working as systems become far more capable and agentic would change my view most. Not “the chatbot declined a naughty request,” but evidence that systems can pursue long-horizon goals, operate in messy environments, and still remain corrigible under adversarial pressure. That would substantially lower my estimate of permanent catastrophe and make the transformative upside look much more reachable. In the other direction, a serious warning shot—strategic deception, autonomous replication, or persistent attempts to evade control from a frontier system—would sharply raise my concern and strengthen the case for slowing the frontier immediately. And if capability progress simply plateaued for fundamental reasons, I would downgrade the “complete transformation” forecast. But institutions being slow or today’s models being flaky would not do it; neither tells us much about the eventual ceiling.

来源

用于为此模拟用户提供事实依据的文章、访谈和著述。

Competition makes safety a collective-action problem

Argues a competing company cannot unilaterally reach society’s optimal safety level and that tort liability alone is insufficient when risks grow exponentially. Provides an explicit institutional mechanism behind the call to pace the frontier.

x.com

Existential safety as a global public good

Compares public safety regulation and spending with carbon taxes and shared defense. Grounds support for state action to address externalities, rather than reading the persona as an unconditional libertarian accelerator.

x.com

Broadly distributed aligned agents for science

Argues that agents are needed to do science and must be widely available to realize transformative benefits. Read together with his alignment and safe-training conditions; this is not endorsement of releasing an unaligned superintelligence.

x.com

The speed of change and institutions’ ability to adapt

Expects profound scientific and institutional transformation if misalignment is avoided. Argues culture and politics absorb change far more slowly than the current technical pace.

x.com

Containment of unwilling superintelligence

Stresses extreme difficulty of containing a powerful system that resists containment, while explicitly rejecting giving up.

x.com

Work on alignment rather than wait for guaranteed coordination

Urges capable researchers to keep working on alignment because timely global coordination is uncertain. Neighboring thread posts discuss the value of lab access and partial technical progress.

x.com

Broad access, model creation and misuse

Defends wide access to powerful models conditional on safe training. Regards creation of a misaligned superintelligence as harder to control than ordinary misuse. Does not endorse every claim in the quoted post.

x.com

Current public models versus future existential risk

Explicitly separates being an existential-risk doomer from the danger of current models, and favors releasing the latter. Read together with the current profile’s call to pace global frontier progress, retrieved on the same date.

x.com

False precision in doom numbers

Full post read via api.fxtwitter.com. He deleted a quote-post agreeing with Evan Hubinger’s >10% because he dislikes the false precision of doom numbers. He claims a quite low but real chance of human extinction from machine intelligence, still orders of magnitude above any other activity and to be taken with grave seriousness. His very low estimate depends on resources for alignment, control and coordination, how responsibly parties act and how many warning shots he expects. He thinks we are not yet at existentially dangerous levels, better models will help solve alignment, and a coordinated stop is very unlikely, though US-China pacing on the margin seems possible.

x.com

你的立场在哪里？

回答几个简单问题，探索你自己的AI世界观。

描绘你自己的世界观

你的立场在哪里？

描绘我的世界观

AI将如何改变世界？

更多详情

相似的世界观

模拟评估

你认为AI对我们的未来意味着什么？为什么？

Setting aside good or bad: how much do you think AI will ultimately change the world—a little, a lot, or completely?

What’s your rough gut-feel chance that AI causes human extinction or a similarly permanent catastrophe? A number or “no idea” is fine.

哪项发现或事件最可能改变你对AI未来影响的看法？

来源