Jack Morris

Jack Morris

x.com/jxmnop

Language model researcher who studies memorization and privacy leaks from text embeddings and writes about reinforcement learning and synthetic data.

AIは世界をどのように変えるでしょうか?

文明規模の変化漸進的な変化DoomBloom
未確定範囲の中心解釈範囲

横軸:その人が表明したDoom–Bloomの見通し。 縦軸:変革の規模。

Doom–Bloom:100点中47。変革の規模:100点中50。解釈範囲:横方向は42から52、縦方向は0から100。これらは解釈上の座標であり、事象の確率ではありません。

Jack MorrisのP(doom) · 推定

≈8%

0%100%

本人が示した数値ではなく、シミュレーションされた本人の回答から推定したものです。 妥当と考えられる範囲:3–19%。

P(doom)の推定方法

その人の見通しを左右するもの

中心的な前提

The key variable for me is how much useful output systems can produce per unit of human input, especially as that input approaches zero.
回答2

この前提が実際には異なると判明した場合、その人の見通しはどう変わりますか?

未解決の問い

So I expect major change, with the ultimate magnitude still unresolved.
回答3

ここで考えられる結果をその人が見分けるうえで、何が役立ちますか?

詳細

予想される恩恵

大きな恩恵が予想されていますが、重要な条件や分配上の制約があります。

67 / 100

影響が小さい変革をもたらす影響

質的尺度での解釈範囲は67から67です。

予想される害

深刻または広範な害が、予想される将来の実質的な一部となっています。

66 / 100

影響が小さい変革をもたらす影響

質的尺度での解釈範囲は67から67です。

人間の影響力

回答に基づく暫定的な推定です。より広い範囲は、ほかにあり得る解釈を示しています。

50 / 100

影響力が小さい影響力が大きい

質的尺度での解釈範囲は0から100です。

これらの解釈では、その人が示した条件が維持されています。恩恵と害は、どちらも大きくなり得ます。この範囲は、統計的な信頼区間ではなく、その人のシミュレーションされた回答をどのように読み取ったかを示すものです。

あなたはJack Morrisと比べてどの位置でしょうか?
約3分で自分のAIに対する世界観をマッピングして、比較できます

似ている世界観

シミュレーションされた世界観がJack Morrisの世界観に最も近いオピニオンリーダー

出典

このシミュレーション対象者の根拠として使用された記事、インタビュー、著作です。

シミュレーション評価

質問1

AIは私たちの未来にとって何を意味すると思いますか?また、なぜそう思いますか?

I think the most useful way to discuss AI’s future is to avoid treating “AGI” as one inevitable threshold. I’d rather measure how much useful output systems produce per unit of human input. That lets us ask concrete questions: Can an agent find important software bugs, conduct productive experiments, or complete work without constant supervision? Coding agents already show practical value in bug-finding, but truly zero-human-input economic production would be a qualitatively more consequential—and potentially frightening—development. The trajectory is also harder to extrapolate than simple scaling stories suggest. Reinforcement learning appears to teach models new ways of using compute, not merely extract behaviors learned during pretraining. Models are also beginning to learn from self-generated data and use memory in interesting ways, although calibrating those memories and generating scalable, useful training data remain open problems. More capable AI researchers may learn from much smaller experiments than humans do, so current estimates of how much compute scientific progress requires could be misleading. The same capabilities create asymmetric risks. Systems that find bugs can help defenders, but capable adversaries can use them to discover vulnerabilities and compromise devices or cloud accounts. Privacy is another concrete concern: models can memorize training data, and even text embeddings can reveal surprisingly much of the original text and personal information. So I expect substantial practical gains alongside serious security and privacy problems, with the outcome depending less on whether we cross a named threshold than on autonomy, human oversight, and measurable capability in specific domains.

質問2

恩恵と害を総合すると、AIは私たちの社会に全体としてどのような影響を与えると予想しますか?

I don’t think I can honestly compress the overall impact into a single positive or negative forecast. The measurable picture is mixed: AI can already help find real software bugs and reduce the human input needed for useful technical work, while the same capability can improve vulnerability discovery for attackers. Likewise, memory and representation learning can make systems more useful, but memorization and embedding inversion create concrete privacy risks. The key variable for me is how much useful output systems can produce per unit of human input, especially as that input approaches zero. If they remain tools that amplify supervised work, the benefits could be substantial and comparatively manageable. If they become reliable, autonomous producers of economic or scientific output—or capable offensive cyber agents—the societal consequences become much larger and potentially frightening. So I expect significant benefits and serious harms, not a clean verdict. I would track autonomy, calibration, security, privacy leakage, and performance on concrete tasks rather than infer society’s future from whether someone declares that “AGI” has arrived.

質問3

AIは最終的に世界をどの程度変えると思いますか?

Potentially enormously, but “how much” depends on measurable autonomy rather than whether we cross an “AGI” threshold. If AI mostly amplifies supervised human work—finding bugs, assisting research, reducing routine effort—that is already a substantial change. If systems can reliably produce valuable economic or scientific output with almost no human input, the change could be qualitatively larger and more destabilizing. I also would not extrapolate too confidently from current bottlenecks. Reinforcement learning can teach models new ways to use compute; self-generated data and memory may broaden learning; and more capable AI researchers might extract useful conclusions from much smaller experiments than humans require. But memory calibration and scalable generation of useful training data remain open problems. So I expect major change, with the ultimate magnitude still unresolved. The most informative indicators are concrete: useful output per human input, reliability over long tasks, scientific productivity, cyber capability, and privacy leakage—not a binary declaration that AGI has arrived.

質問4

AIが人類絶滅、または同様に永続的な破局を引き起こす可能性は、直感的にどのくらいだと思いますか?

I don’t have a defensible numerical probability. My concrete concerns are about increasingly autonomous systems, AI-enabled cyber intrusion, privacy leakage, and the possibility of near-zero-human-input economic or scientific production. Those could become severe, but they do not by themselves establish an extinction forecast. I’d rather track measurable autonomy, reliability, offensive capability, and the amount of useful output produced without human oversight than invent a precise catastrophe percentage.
あなたはどの位置でしょうか?
いくつかの簡単な質問に答えて、自分のAIに対する世界観を探ってみましょう。
自分の世界観をマッピングする

あなたはどの位置でしょうか?

自分の世界観をマッピングする