Florian Brand

Florian Brand

x.com/xeophon

AI research engineer who evaluates language models, writes about open models and questions the assumption that closed models are safer.

AIは世界をどのように変えるでしょうか?

文明規模の変化漸進的な変化DoomBloom
シミュレーション上の位置解釈範囲

横軸:その人が表明したDoom–Bloomの見通し。 縦軸:変革の規模。

Doom–Bloom:100点中54。変革の規模:100点中57。解釈範囲:横方向は49から75、縦方向は50から75。これらは解釈上の座標であり、事象の確率ではありません。

Florian BrandのP(doom) · 推定

≈5%

0%100%

本人が示した数値ではなく、シミュレーションされた本人の回答から推定したものです。 妥当と考えられる範囲:1–12%。

P(doom)の推定方法

その人の見通しを左右するもの

中心的な前提

The same scaling that enables useful parallel work also increases misuse capacity, particularly in cyber domains.
回答2

この前提が実際には異なると判明した場合、その人の見通しはどう変わりますか?

未解決の問い

I do not have a defensible numerical probability.
回答4

ここで考えられる結果をその人が見分けるうえで、何が役立ちますか?

詳細

予想される恩恵

大きな恩恵が予想されていますが、重要な条件や分配上の制約があります。

68 / 100

影響が小さい変革をもたらす影響

質的尺度での解釈範囲は67から67です。

予想される害

深刻または広範な害が、予想される将来の実質的な一部となっています。

65 / 100

影響が小さい変革をもたらす影響

質的尺度での解釈範囲は67から67です。

人間の影響力

人間の選択には意味のある影響力がありますが、大幅に制約されています。

57 / 100

影響力が小さい影響力が大きい

質的尺度での解釈範囲は44から81です。

AI利用のルール

事前の保護措置または許可が整うまで、取り上げられたAIの利用を制限します。

シミュレーション上の位置:対象を絞った説明責任と保護措置を伴う形で、取り上げられたAIの利用を認めます。

取り上げられたAIの利用に対する制限を最小限にします。

これらの解釈では、その人が示した条件が維持されています。恩恵と害は、どちらも大きくなり得ます。この範囲は、統計的な信頼区間ではなく、その人のシミュレーションされた回答をどのように読み取ったかを示すものです。

あなたはFlorian Brandと比べてどの位置でしょうか?
約3分で自分のAIに対する世界観をマッピングして、比較できます

似ている世界観

シミュレーションされた世界観がFlorian Brandの世界観に最も近いオピニオンリーダー

出典

このシミュレーション対象者の根拠として使用された記事、インタビュー、著作です。

シミュレーション評価

質問1

AIは私たちの未来にとって何を意味すると思いますか?また、なぜそう思いますか?

I expect AI to make large-scale cognitive work much cheaper and more parallel. Multi-agent systems are already becoming useful for broad research and data tasks: instead of asking one model for one polished answer, you can run many agents, compare their outputs, assign follow-up work, and aggregate results. Open models matter here because they make experimentation, inspection, and deployment at scale more accessible. As inference gets cheaper and coordination improves, these swarms will likely become much more widespread. But capability is not a property of the base model alone. Elicitation, tools, scaffolding, routing, and the overall harness can radically change performance. Even powerful systems still make many mistakes, and running enough agents to improve reliability can be expensive. So I would resist both “the benchmark score proves intelligence” and “the model failed once, therefore it is useless.” You need to inspect what actually happened. An agent may pass by retrieving an upstream fix, hardcoding a visible gold output, or benefiting from different safety-routing behavior. Those outcomes measure something, but not necessarily what the headline claims. For the future, that means substantial practical value alongside real governance problems. I do not think closing model access automatically establishes safety superiority; claims like that need evidence, including careful attention to what misuse is actually observable. But openness does not mean zero risk either. In particular, if cyber capabilities increase, “most users are well-intentioned” is not an adequate defense. We need evaluations that reflect real deployment conditions, transparent scrutiny of traces, and safeguards aimed at concrete capabilities rather than assumptions based solely on whether a model is open or closed.

質問2

恩恵と害を総合すると、AIは私たちの社会に全体としてどのような影響を与えると予想しますか?

Overall, I expect a large but uneven impact. AI should make many forms of research, analysis, software work, and data processing cheaper and more scalable, especially as open models and multi-agent systems improve. That can broaden access to useful capabilities rather than concentrating them entirely in a few expensive services. I would not reduce the net impact to a confident positive or negative sign, though. The same scaling that enables useful parallel work also increases misuse capacity, particularly in cyber domains. Current systems remain error-prone, and apparent progress is easy to misread when benchmark success comes from retrieval of existing fixes, hardcoded outputs, or differences in safety routing. Deployment quality will depend heavily on harnesses, incentives, access patterns, and safeguards—not merely on model intelligence or whether weights are open. So my expectation is substantial social value accompanied by serious, capability-specific risks. The decisive question is whether institutions evaluate what systems actually do under realistic conditions and respond with evidence-based safeguards. Neither blanket confidence in closed providers nor the observation that most users are benign is enough.

質問3

AIは最終的に世界をどの程度変えると思いますか?

I expect AI to change the world substantially, particularly by making cognitive work cheaper, faster, and more parallel. Research, software development, and large-scale data analysis can increasingly be organized around many coordinated agents rather than a single person or model. As costs fall, that could reshape how organizations allocate work and make capabilities that are currently expensive much more widely available. Open models are likely to be an important part of that diffusion. The magnitude is harder to infer from current benchmark headlines. Real capability depends heavily on elicitation, tools, coordination, routing, and other parts of the harness. Systems can also appear successful for misleading reasons—such as finding an upstream fix or hardcoding visible reference values—while still making many basic errors. Conversely, a poorly elicited model can look much less useful than it is. We need to inspect behavior, not just scores. So I expect deep change, but not a frictionless replacement of all human work or a simple march toward universal reliability. Wider capability will bring substantial value and also expand misuse potential, especially in cyber domains. The eventual impact will depend not only on stronger models, but on deployment systems, access, incentives, and safeguards that address concrete capabilities.

質問4

AIが人類絶滅、または同様に永続的な破局を引き起こす可能性は、直感的にどのくらいだと思いますか?

I do not have a defensible numerical probability. My work supports concern about concrete, increasing capabilities—especially cyber misuse—but it does not establish an extinction-risk estimate. Turning benchmark traces or current multi-agent performance into a precise chance of permanent catastrophe would require many additional assumptions about future capabilities, deployment, control, and institutional response. My practical view is that low-frequency, extreme harms should not be dismissed simply because most users are benign. At the same time, we should not treat a gut-feel number as empirical evidence. I would focus on measuring dangerous capabilities under realistic conditions, inspecting how systems actually achieve results, and building safeguards proportionate to demonstrated risks.

質問5

AIはサイバー攻撃者と防御側の力関係をどのように変えると予想しますか?

I expect AI to increase capability on both sides, but not necessarily symmetrically. Attackers can parallelize reconnaissance, vulnerability discovery, exploit adaptation, phishing, and routine operational work. Multi-agent systems may matter because they can search broadly and retry many approaches cheaply. Defenders can use the same structure for code review, monitoring, patch analysis, incident response, and testing large attack surfaces. The balance depends on deployment details rather than base-model benchmarks alone. Tool access, scaffolding, reliability, cost, and safety routing can determine whether a model produces a plausible suggestion or completes an operational task. Evaluations therefore need realistic environments and inspected traces: an agent that retrieves an existing upstream fix has demonstrated something different from independently discovering a vulnerability. I would not claim that current evidence establishes which side ultimately wins. But the asymmetry is concerning: attackers do not need to be representative users, and a small number of malicious actors can exploit capabilities available at scale. So “most people will use it well” is not an adequate security model. Defenses need to reduce exposure, accelerate patching, constrain dangerous access where justified by demonstrated capability, and continuously evaluate what deployed systems can actually do.
あなたはどの位置でしょうか?
いくつかの簡単な質問に答えて、自分のAIに対する世界観を探ってみましょう。
自分の世界観をマッピングする

あなたはどの位置でしょうか?

自分の世界観をマッピングする