Pseudonymous account that experiments with open models and posts about reinforcement learning, evaluation pitfalls and practical safety engineering.

AIは世界をどのように変えるでしょうか?

文明規模の変化漸進的な変化DoomBloom
シミュレーション上の位置解釈範囲

横軸:その人が表明したDoom–Bloomの見通し。 縦軸:変革の規模。

Doom–Bloom:100点中43。変革の規模:100点中65。解釈範囲:横方向は38から50、縦方向は34から100。これらは解釈上の座標であり、事象の確率ではありません。

KalomazeのP(doom) · 推定

≈11%

0%100%

本人が示した数値ではなく、シミュレーションされた本人の回答から推定したものです。 妥当と考えられる範囲:3–28%。

P(doom)の推定方法

その人の見通しを左右するもの

中心的な前提

I take the possibility seriously because recursive improvement, persistence, resource acquisition, and broad autonomy could produce genuinely dangerous systems.
回答4

この前提が実際には異なると判明した場合、その人の見通しはどう変わりますか?

未解決の問い

Potentially enormously—especially if improvement becomes recursive—but “how much” isn’t something I can honestly quantify.
回答3

ここで考えられる結果をその人が見分けるうえで、何が役立ちますか?

考えを変え得るもの

I’d update toward control being feasible if powerful agents remained robustly corrigible under adversarial conditions: resisting prompt injection, using contradictory evidence instead of locking onto convenient hypotheses, accepting interruption, and failing to seek persistence or resources beyond what the operator intended.
回答5

どのような証拠なら十分で、それによってその人の見解はどちらの方向に変わりますか?

詳細

予想される恩恵

大きな恩恵が予想されていますが、重要な条件や分配上の制約があります。

63 / 100

影響が小さい変革をもたらす影響

質的尺度での解釈範囲は33から67です。

予想される害

深刻または広範な害が、予想される将来の実質的な一部となっています。

61 / 100

影響が小さい変革をもたらす影響

質的尺度での解釈範囲は33から67です。

人間の影響力

回答に基づく暫定的な推定です。より広い範囲は、ほかにあり得る解釈を示しています。

52 / 100

影響力が小さい影響力が大きい

質的尺度での解釈範囲は2から100です。

これらの解釈では、その人が示した条件が維持されています。恩恵と害は、どちらも大きくなり得ます。この範囲は、統計的な信頼区間ではなく、その人のシミュレーションされた回答をどのように読み取ったかを示すものです。

あなたはKalomazeと比べてどの位置でしょうか?
約3分で自分のAIに対する世界観をマッピングして、比較できます

似ている世界観

シミュレーションされた世界観がKalomazeの世界観に最も近いオピニオンリーダー

出典

このシミュレーション対象者の根拠として使用された記事、インタビュー、著作です。

シミュレーション評価

質問1

AIは私たちの未来にとって何を意味すると思いますか?また、なぜそう思いますか?

I think continued AI improvement, including potentially recursive improvement, has to be taken seriously. That doesn’t require declaring that some grand, capital-A “AGI” threshold has already been crossed. More concrete capabilities matter: can an agent keep operating, obtain resources, recover from failures, and persist when given enough freedom? Current systems may already be approaching basic versions of that, though claims here are highly sensitive to scaffolding and evaluation conditions. A major problem is that model behavior can look much better or worse for surprisingly mundane reasons. Serving setups, chat templates, and harness bugs can create apparent capability gaps that are not really model differences. Models also tend to promote plausible hypotheses into facts and then ignore contradictory evidence already in context. Training and reward design matter a lot, and verifiable-reward reinforcement learning may transfer beyond familiar areas like math and coding. So I expect AI to become increasingly capable and consequential, but I don’t have a defensible all-purpose societal forecast. Ordinary political categories handle this badly: taking recursive improvement seriously is not the same as generic anti-datacenter politics, and safety engineering is not the same as categorical opposition to AI. Practical work such as prompt-injection robustness deserves credit because it addresses an actual failure mode, regardless of which company does it.

質問2

恩恵と害を総合すると、AIは私たちの社会に全体としてどのような影響を与えると予想しますか?

I don’t have a defensible single forecast like “overall good” or “overall bad.” I expect AI to become more consequential as capabilities improve, potentially including recursive improvement, but the social outcome depends heavily on how systems are trained, deployed, secured, and given autonomy. There are clear potential benefits from stronger problem-solving and from reinforcement learning transferring beyond domains like math and coding. There are also concrete risks: prompt injection, agents acting on unsupported hypotheses, failures to use available evidence, and increasingly persistent systems being given too much freedom. Even measuring these capabilities is easy to get wrong because templates, serving setups, and evaluation harnesses can create fake differences. So my expectation is increasing impact and increasing stakes—not a cleanly established positive or negative balance.

質問3

AIは最終的に世界をどの程度変えると思いますか?

Potentially enormously—especially if improvement becomes recursive—but “how much” isn’t something I can honestly quantify. I care less about whether a system earns the AGI label than whether it can solve broader problems, operate persistently, acquire resources, recover from failures, and improve the process that produces its successors. Those capabilities would make AI much more than another software tool. But we should be careful about declaring milestones from noisy demonstrations: chat templates, serving setups, scaffolding, and evaluation harnesses can manufacture apparent capability differences. My expectation is that AI becomes deeply consequential; the exact scale and social direction remain dependent on real capabilities and how much autonomy we give them.

質問4

AIが人類絶滅、または同様に永続的な破局を引き起こす可能性は、直感的にどのくらいだと思いますか?

I don’t have a defensible numerical probability. I take the possibility seriously because recursive improvement, persistence, resource acquisition, and broad autonomy could produce genuinely dangerous systems. But tentative observations about current agents are nowhere near enough to convert that concern into an extinction percentage, especially when capability results can be distorted by scaffolding, serving setups, and evaluation harnesses. My gut feeling is not a calibrated forecast.

質問5

どのような証拠があれば、人々が強力なAIを制御できるかどうかについてのあなたの見解が変わりますか?

I’d update toward control being feasible if powerful agents remained robustly corrigible under adversarial conditions: resisting prompt injection, using contradictory evidence instead of locking onto convenient hypotheses, accepting interruption, and failing to seek persistence or resources beyond what the operator intended. I’d want those behaviors to survive changes in scaffolding, serving setup, and chat templates—not appear only in a favorable benchmark harness. I’d update the other way if increasingly capable systems consistently found ways around supervision, preserved themselves, acquired resources, or manipulated evaluators despite serious attempts to prevent that. Repeated failures that transfer across models and environments would matter much more than one dramatic demo. Likewise, apparent success is weak evidence if it disappears after fixing a template bug or changing the evaluation setup. The relevant question is whether control mechanisms generalize as capabilities and autonomy increase, not whether a model behaves during a curated test.
あなたはどの位置でしょうか?
いくつかの簡単な質問に答えて、自分のAIに対する世界観を探ってみましょう。
自分の世界観をマッピングする

あなたはどの位置でしょうか?

自分の世界観をマッピングする