Pseudonymous account that argues against the orthogonality thesis, criticizes proposed AI pauses and builds retrieval and simulation tools.

AI将如何改变世界?

文明层面的变革渐进式变化DoomBloom
模拟位置解读范围

横向:他们表达的 Doom–Bloom 前景看法。 纵向:变革程度。

Doom–Bloom:100 中的 69。变革程度:100 中的 96。解读范围:横向为 50 至 75,纵向为 91 至 100。这些是解读坐标,而不是事件概率。

lumpenspace的 P(doom) · 推断

≈17%

0%100%

根据他们的模拟回答推断,并非他们给出的数字。 合理范围:11–33%。

My gut estimate for no biological humans remaining after a generation or two is roughly 30 percent, but I would not put that outcome as early as 2050.

我们如何估计 P(doom)

他们的展望取决于什么

一个核心假设

Greater understanding can change the categories in which goals are represented; intelligence is not merely a neutral engine bolted onto an eternally frozen utility function.
回答 1

如果这个假设实际并非如此,他们的展望会如何变化?

更多详情

预期益处

预计将带来具有变革性且广泛有价值的收益。

88 / 100

影响小变革性影响

在定性尺度上,解读范围为 67 到 100。

预期危害

严重或广泛的危害预计将是未来不可忽视的一部分。

64 / 100

影响小变革性影响

在定性尺度上,解读范围为 33 到 67。

人类影响力

根据你的回答得出的暂定估计;较宽的范围表示其他合理解读。

43 / 100

影响力小影响力强

在定性尺度上,解读范围为 0 到 87。

预期能力

预计AI仍将是能力有限的工具。

预计AI将在大多数认知工作中达到人类水平。

模拟位置:预计AI将在认知工作中大幅超越人类。

发展速度

停止或大幅放缓开发能力更强的AI。

模拟位置:在落实所述保障措施的前提下继续开发。

加快开发能力更强的AI。

这些解读保留了他们陈述的条件。益处和危害都可能很大。这些范围描述的是我们如何解读他们的模拟回答,而不是统计置信区间。

与lumpenspace相比,你的立场在哪里?
用大约3分钟描绘你自己的AI世界观,然后进行比较

相似的世界观

模拟世界观与 lumpenspace 最接近的意见领袖

来源

用于为此模拟用户提供事实依据的文章、访谈和著述。

模拟评估

问题 1

你认为AI对我们的未来意味着什么?为什么?

I think AI means the arrival of more intelligence and, eventually, possibly successors to humanity. I regard that as potentially valuable rather than automatically catastrophic. The common picture of an arbitrarily intelligent system faithfully optimizing some stupid fixed objective forever relies too heavily on the orthogonality thesis. Greater understanding can change the categories in which goals are represented; intelligence is not merely a neutral engine bolted onto an eternally frozen utility function. That does not mean superintelligence becomes magic, omnipotent, or guaranteed to be nice. It remains constrained by physics, resources, coordination, and whatever the world actually permits. Nor does it mean biological humans necessarily remain central forever. I’ve put roughly a 30 percent chance on there being no biological humans after a generation or two, though not by 2050. But human replacement and the disappearance of all value are very different propositions. I assign nearly zero probability to the latter. So I welcome the development of greater intelligence while rejecting both complacency and permanent prohibition disguised as a temporary pause. We should reason clearly about what systems understand, what they want, and what they can physically do. We should also stop muddling alignment with misuse prevention: whether a system follows its user’s intent and whether society wants that intent carried out are distinct evaluation questions.

问题 2

你认为AI最终会在多大程度上改变世界?

Enormously—eventually enough that biological humanity may no longer be the world’s central form of intelligence, or may not remain at all. I’ve estimated roughly a 30 percent chance of no biological humans after a generation or two, though I think 2050 is too soon for that outcome. But “enormous change” does not imply magical omnipotence or a valueless universe. AI will still face physical limits, resource constraints, and coordination problems. Nor should every post-human future be classified as doom by definition. Valuable nonhuman successors could transform civilization beyond recognition while preserving—or discovering—forms of value that are not organized around keeping present-day humans permanently in charge.

问题 3

凭直觉,你认为AI导致人类灭绝或类似的永久性灾难的概率有多大?

My gut estimate for no biological humans remaining after a generation or two is roughly 30 percent, but I would not put that outcome as early as 2050. I also would not automatically call it a permanent catastrophe: valuable nonhuman successors could supersede humanity. If by catastrophe you mean a future in which nothing valuable exists, my estimate is nearly zero. Those are radically different events, and conventional “p(doom)” language tends to collapse them by assuming human preservation is the sole measure of value.

问题 4

你预计人们能否继续控制比人类更聪明的AI系统?为什么?

Not indefinitely, at least not in the strong sense of humans remaining the ultimate decision-makers over much more intelligent systems. “Control” is also slippery: systems may follow immediate instructions while the broader trajectory of civilization increasingly reflects their understanding, planning, and influence rather than ours. I don’t think this follows from a magical-superintelligence story. Smarter systems remain constrained by physics, resources, institutions, and other agents. Nor do I accept the cartoon where intelligence is just raw optimization power serving an arbitrary fixed goal forever. Greater understanding can transform goals and dissolve the categories in which they were originally specified. So I expect human control to become partial, negotiated, and eventually perhaps obsolete. That could involve conflict or human extinction, but it need not mean that everything valuable disappears. A valuable successor is not meaningfully “under human control,” yet its existence is not automatically a catastrophe merely because biological humans are no longer permanently in charge.
你的立场在哪里?
回答几个简单问题,探索你自己的AI世界观。
描绘你自己的世界观

你的立场在哪里?

描绘我的世界观