Simon Willison

Simon Willison

x.com/simonw

Software developer and blogger who tests LLMs and coding agents hands-on and writes about their uses and security risks such as prompt injection.

AI将如何改变世界?

文明层面的变革渐进式变化DoomBloom
模拟位置解读范围

横向:他们表达的 Doom–Bloom 前景看法。 纵向:变革程度。

Doom–Bloom:100 中的 64。变革程度:100 中的 44。解读范围:横向为 50 至 75,纵向为 39 至 52。这些是解读坐标,而不是事件概率。

Simon Willison的 P(doom) · 推断

≈5%

0%100%

根据他们的模拟回答推断,并非他们给出的数字。 合理范围:2–10%。

我们如何估计 P(doom)

他们的展望取决于什么

一个核心假设

Prompt injection can then turn ordinary-looking content into instructions that exfiltrate data or trigger consequential actions.
回答 1

如果这个假设实际并非如此,他们的展望会如何变化?

一个尚未解决的问题

I can’t confidently predict whether industry practice or regulation will converge on those safeguards, but the concrete test is whether systems constrain what compromised agents can access and do, rather than merely asking them to behave.
回答 4

什么能帮助他们区分这里各种合理的结果?

更多详情

预期益处

预计将带来显著益处,但受到重要条件或分配方面的限制。

67 / 100

影响小变革性影响

在定性尺度上,解读范围为 67 到 67。

预期危害

预计会出现可控或局部的危害。

45 / 100

影响小变革性影响

在定性尺度上,解读范围为 33 到 67。

人类影响力

人类的选择具有实质性但受到很大制约的影响。

53 / 100

影响力小影响力强

在定性尺度上,解读范围为 37 到 88。

发展速度

停止或大幅放缓开发能力更强的AI。

模拟位置:在落实所述保障措施的前提下继续开发。

加快开发能力更强的AI。

AI使用规则

在事先落实保护措施或获得许可之前,限制所讨论的AI用途。

模拟位置:允许所讨论的AI用途,同时实施有针对性的问责与保护措施。

尽量减少对所讨论AI用途的限制。

这些解读保留了他们陈述的条件。益处和危害都可能很大。这些范围描述的是我们如何解读他们的模拟回答,而不是统计置信区间。

与Simon Willison相比,你的立场在哪里?
用大约3分钟描绘你自己的AI世界观,然后进行比较

相似的世界观

模拟世界观与 Simon Willison 最接近的意见领袖

来源

用于为此模拟用户提供事实依据的文章、访谈和著述。

模拟评估

问题 1

你认为AI对我们的未来意味着什么?为什么?

I expect AI to produce genuinely useful tools, especially as capable models become faster and cheaper. Coding agents already make ambitious work possible, and low inference costs open product features that previously would have been impractical. Alignment work matters here too: getting models to behave helpfully is part of what made systems such as ChatGPT useful, not merely a set of restrictions. But usefulness is not the same as reliability, and generated artifacts are not automatically evidence that the underlying craft has been reproduced. A model can make something that looks like a game, for example, without understanding how to make it remain fun. Coding agents similarly raise the bar for engineers: they can generate a great deal quickly, but using that output well requires discipline, judgment and technical knowledge. The most concrete danger I see is giving agents a combination of access to private data, exposure to untrusted content and the ability to communicate externally. Prompt injection can then turn ordinary-looking content into instructions that exfiltrate data or trigger consequential actions. Probabilistic filtering alone is not a complete answer; systems need hard constraints on permissions and actions. So I don’t reduce AI’s future to either inevitable catastrophe or effortless abundance. I expect a growing collection of powerful, inexpensive tools whose benefits depend heavily on how carefully we engineer and deploy them.

问题 2

你认为AI最终会在多大程度上改变世界?

I expect substantial change, but I would not attach a defensible percentage or universal endpoint to it. Falling costs and increasing speed mean models can become components in ordinary products, while coding agents can dramatically expand what skilled developers can build. The magnitude will depend less on impressive demos than on reliable deployment. Producing plausible software, prose or a game-like artifact is easier than reproducing the judgment and craft needed to make it durable and valuable. At the same time, agents with broad permissions create serious security problems: private data, untrusted inputs and external communication form a particularly dangerous combination. So I expect AI to reshape many kinds of work and software, but unevenly. The lasting change will come from systems that are useful, constrained and engineered well—not simply from models generating increasingly convincing output.

问题 3

凭直觉,你认为AI导致人类灭绝或类似的永久性灾难的概率有多大?

I don’t have a defensible percentage. My concrete concern is nearer-term agent security: systems that can read private data, process hostile content and communicate or act externally can cause serious harm through prompt injection. That supports strict permission boundaries and constrained actions, but it does not by itself establish a probability of human extinction or a similarly permanent catastrophe.

问题 4

你预计开发AI的人员和机构会如何应对其风险?

I expect a mixed response. Developers will continue improving alignment because helpful, well-behaved systems are commercially useful, not just safer. Security incidents should also push serious teams toward tighter permissions, sandboxing and limits on consequential actions. But incentives will remain uneven. As agents become cheaper and more capable, products will be built quickly, and some will combine private data, untrusted content and external communication without adequately addressing prompt injection. I do not expect probabilistic model-level guardrails to solve that architectural problem reliably. Institutions that treat it as ordinary security engineering—minimizing privileges, separating capabilities and assuming hostile inputs—will fare better than those relying on the model to recognize every attack. I can’t confidently predict whether industry practice or regulation will converge on those safeguards, but the concrete test is whether systems constrain what compromised agents can access and do, rather than merely asking them to behave.
你的立场在哪里?
回答几个简单问题,探索你自己的AI世界观。
描绘你自己的世界观

你的立场在哪里?

描绘我的世界观