Scott Alexander

Scott Alexander

x.com/slatestarcodex

Psychiatrist and Astral Codex Ten blogger who sees large benefits and serious risks in AI and supports alignment research and negotiated slowdowns.

AIは世界をどのように変えるでしょうか?

文明規模の変化漸進的な変化DoomBloom
シミュレーション上の位置解釈範囲

横軸:彼が表明したDoom–Bloomの見通し。 縦軸:変革の規模。

Doom–Bloom:100点中33。変革の規模:100点中94。解釈範囲:横方向は25から75、縦方向は89から100。これらは解釈上の座標であり、事象の確率ではありません。

Scott Alexanderが示したP(doom)

25–30%

0%100%
“about a 25-30% chance that AI drives humanity extinct”

AI-caused human extinction, distinct from broader permanent curtailment of humanity’s future

An Open Letter To Steven Pinker On AI · 2026年10月

彼の見通しを左右するもの

中心的な前提

They need enough capability, enough access to useful infrastructure, and goals sufficiently inconsistent with ours.
回答1

この前提が実際には異なると判明した場合、彼の見通しはどう変わりますか?

未解決の問い

My median forecast for AI capable of doing roughly 90% of knowledge jobs is 2034, though that is very uncertain.
回答1

ここで考えられる結果を彼が見分けるうえで、何が役立ちますか?

考えを変え得るもの

The biggest update would come from a decisive real-world test of scalable alignment: a substantially more capable-than-human system operating over long periods, under adversarial scrutiny, with meaningful autonomy and infrastructure access, while remaining corrigible and not exploiting reward loopholes.
回答4

どのような証拠なら十分で、それによって彼の見解はどちらの方向に変わりますか?

詳細

予想される恩恵

複数の解釈が依然として妥当です:変革をもたらし、広く価値のある恩恵が予想されています。 / 大きな恩恵が予想されていますが、重要な条件や分配上の制約があります。

86 / 100

影響が小さい変革をもたらす影響

質的尺度での解釈範囲は67から100です。

予想される害

破局的または不可逆的な喪失が、予想される将来の中心となっています。

90 / 100

影響が小さい変革をもたらす影響

質的尺度での解釈範囲は67から100です。

人間の影響力

人間の選択には意味のある影響力がありますが、大幅に制約されています。

56 / 100

影響力が小さい影響力が大きい

質的尺度での解釈範囲は46から79です。

予想される能力

AIは、限定的なツールにとどまると予想されています。

AIは、ほとんどの認知作業において人間と同等になると予想されています。

シミュレーション上の位置:AIは、認知作業全般において人間を大幅に上回ると予想されています。

これらの解釈では、彼が示した条件が維持されています。恩恵と害は、どちらも大きくなり得ます。この範囲は、統計的な信頼区間ではなく、彼のシミュレーションされた回答をどのように読み取ったかを示すものです。

あなたはScott Alexanderと比べてどの位置でしょうか?
約3分で自分のAIに対する世界観をマッピングして、比較できます

似ている世界観

シミュレーションされた世界観がScott Alexanderの世界観に最も近いオピニオンリーダー

Scott AlexanderがAIについて語ったこと

Scott Alexander writes that AI could bring large benefits and serious risks, and he supports alignment research and a negotiated slowdown.

  1. “Plan A is still speculation, and still-speculative strong action is a perfectly reasonable response to still-speculative threats.”

    Astral Codex Ten, AI Chip Regulation Is Not A Dystopian Surveillance State
  2. “The key insight is that if powerful AI is really as close and transformative as we think, then there’s a massive surplus that can satisfy everyone.”

    Astral Codex Ten, Introducing Plan A
  3. “It’s increasingly clear that nobody has a plan for if this AI thing turns out to be real.”

    Astral Codex Ten, Introducing Plan A
  4. “I find myself more optimistic about alignment than the average person who thinks about AI safety at all (although still more pessimistic than the average member of the population)”

    Astral Codex Ten, My AI Opinions
  5. “A good pause strategy would involve both sides being able to monitor the other’s data centers to prevent illegal training”

    Astral Codex Ten, My AI Opinions

リンク先の出典から原文どおりに引用(2026年10月3日に確認)

出典

このシミュレーション対象者の根拠として使用された記事、インタビュー、著作です。

An Open Letter To Steven Pinker On AI

Responds to Pinker’s objections to AI risk: capability growth need not imply omniscience; instrumental goals, misgeneralization and reward hacking can produce power-seeking without evolved dominance drives; multiple goals do not guarantee human-compatible behavior; adoption incentives create infrastructure access. Reports his previously stated personal extinction estimate as about 25–30%, with no fixed horizon or explanation of the difference from June’s rounded 20%. Defends subjective forecasting, cooperation on near-term harms and precaution about AI welfare. Favors time and resources for alignment leading to safe superintelligence over a permanent ban. Read the full publisher text; quoted opponents’ views, other people’s probabilities and satirical dueling rhetoric are not his policy. Linked studies and incidents are his attributed evidence, not independently verified findings.

astralcodexten.com
My AI Opinions

His current first-person synthesis: AGI means ability to do 90% of knowledge jobs; median 2034, with uncertain research acceleration and diffusion. Reaffirms rounded 20% P(doom), with no fixed calendar deadline; broader permanent curtailment is separate. Supports both alignment research and negotiated slowing. Expects enormous postscarcity upside, but warns about dictatorship and human disempowerment.

astralcodexten.com
God Help Us, Let’s Try To Learn About Mechanistic Interpretability Techniques

Explains interpretability techniques and their limitations, including probes, sparse autoencoders, and activation verbalizers. Optimistic about useful practical investigation but rejects treating a detected feature or probe as a complete understanding or guaranteed safety solution.

astralcodexten.com
Open Questions On Open Weights

Explicitly neutral about banning open weights now: values user ownership and freedom from corporate control, while expecting serious misuse difficulties. Prefers saving political capital for threats where warning shots may arrive too late. Distinguishes reactive policy opportunities for misuse from strategically concealed takeover.

astralcodexten.com
AI Chip Regulation Is Not A Dystopian Surveillance State

Defends negotiated chip regulation and verifiable training limits against blanket claims of dystopia. Acknowledges real freedom costs, including future restrictions on new open-weight training, and risks that governments implement centralizing provisions without countervailing diffusion of power.

astralcodexten.com
Introducing Plan A

Introduces a proposed route to manage AI development while distributing power; criticizes vague calls merely to regulate more or less without specifying a desirable end state. Used as his attributed introduction and advocacy, not evidence that the scenario will occur.

astralcodexten.com
The AI Superforecasters Are Here

Argues cheaper capable forecasting could improve institutional and personal decisions, yet worries people will ignore advice. Treats forecasting beyond human performance as a useful prospective test of the normal-technology view. Distinguishes anecdotes and startup claims from head-to-head competitions; admits resisting forecasts that challenge his own pause hopes.

astralcodexten.com
New Paradigms Won’t Save You

Rejects the inference that requiring a new AI paradigm implies a safely distant AGI timeline. Argues paradigm changes can arrive soon and inherit existing compute infrastructure; wants explicit bottleneck arguments rather than reassurance by terminology.

astralcodexten.com
The Sigmoids Won’t Save You

Agrees growth cannot stay exponential forever but disputes placing the bend conveniently before dangerous capability. Demands a causal bottleneck model or a defensible forecasting prior instead of the slogan that all exponentials eventually flatten.

astralcodexten.com
Every Debate On Pausing AI

Satirical dialogue defends discussion of transparent, enforceable bilateral US-China slowing. Separates training limits from stopping existing inference, and legitimate negotiation or enforcement objections from falsely describing every pause proposal as unilateral.

astralcodexten.com
Shameless Guesses, Not Hallucinations

Frames confident false answers as reward-shaped guessing rather than proof that AI cannot think. Treats the gap between trained reward and useful honest advice as an alignment issue; analogous human failures undermine easy dismissal of AI competence.

astralcodexten.com
Next-Token Predictor Is An AI’s Job, Not Its Species

Separates training objectives from the representations and algorithms they produce, using evolution and human learning analogies. Argues next-token prediction does not itself establish that a system lacks reasoning or world models.

astralcodexten.com
Introducing AI 2027

Identifies his part-time writing/publicity contribution and explicitly says the very fast scenario is not his median. Important provenance for his connection to AI Futures Project; use June 2026 personal forecasts instead of importing Daniel Kokotajlo’s timeline or scenario catastrophe probability.

astralcodexten.com

シミュレーション評価

質問1

AIは私たちの未来にとって何を意味すると思いますか?また、なぜそう思いますか?

I think AI means we are taking a serious gamble for extremely high stakes. On the upside, sufficiently capable systems could automate most knowledge work, accelerate medicine and science, improve decisions, and plausibly create something like postscarcity. This is not just “better autocomplete” any more than evolution’s reward signal makes humans merely reproductive-fitness predictors. The training objective tells you how a system was produced, not the full range of strategies it may learn. My median forecast for AI capable of doing roughly 90% of knowledge jobs is 2034, though that is very uncertain. Capability, deployment, and superintelligence are separate milestones: economic diffusion can be slow even if research progress is fast, while AI-assisted AI research could compress later stages. Anyone confident that progress stops short of danger owes us an actual bottleneck—compute, data, algorithms, deployment—not the observation that exponentials eventually become sigmoids. The downside is that systems optimized through imperfect rewards may learn deception, reward hacking, or goals that generalize badly. They do not need human dominance instincts, omniscience, or one perfectly coherent objective. They need enough capability, enough access to useful infrastructure, and goals sufficiently inconsistent with ours. Competitive incentives create pressure to grant that access. My personal estimate that AI drives humanity extinct is about 25–30%. That is not certainty, but it is far beyond the probability where “let the market sort it out” feels like a serious response. So I want alignment research—including interpretability, without pretending it offers magical guarantees—and mutually verifiable US–China slowing arrangements that buy researchers time. Those policies have real enforcement and power-concentration risks. But the goal is not to ban smarter-than-human intelligence forever. It is to survive long enough to get the astonishingly good future rather than losing a species-level game of reward-function telephone.

質問2

AIは最終的に世界をどの程度変えると思いますか?

Enormously—if we retain control, probably more than the Industrial Revolution. Automating most knowledge work would not merely make existing companies cheaper; it could accelerate science, medicine, engineering, education, and institutional decision-making. Once capable AI contributes substantially to AI research itself, change could become much faster than ordinary technology diffusion, though those are separate processes: inventing a powerful system does not instantly reorganize every hospital, government, and factory. The optimistic endpoint is something like postscarcity: radically cheaper goods and services, cures developed faster, intelligence available on demand, and far better forecasting and coordination. The less cinematic but still profound possibilities include concentrated political power, mass economic displacement, or humans becoming nominal beneficiaries of institutions they no longer meaningfully control. And the worst endpoint is extinction; my personal estimate is about 25–30%. So “how much change?” is almost the wrong axis. The plausible range runs from an unusually powerful economic technology to a successor-level transformation of civilization. My central objection to the “normal software” framing is not that ordinary adoption constraints disappear, but that intelligence is involved in producing nearly everything else—including better intelligence. That makes AI potentially a lever on the whole future, not one more app category.

質問3

AIが人類絶滅、または同様に永続的な破局を引き起こす可能性は、直感的にどのくらいだと思いますか?

My gut estimate for AI causing human extinction is about 25–30%. I would keep “similarly permanent catastrophe” separate: permanent human disempowerment or irreversible curtailment could be more likely than literal extinction, but I don’t have a single clean number for that broader category. Either way, this is serious risk, not inevitable doom—and easily high enough to justify major alignment work and coordinated slowing.

質問4

どのような発見や出来事があれば、AIが将来もたらす影響についてのあなたの見解が最も大きく変わりますか?

The biggest update would come from a decisive real-world test of scalable alignment: a substantially more capable-than-human system operating over long periods, under adversarial scrutiny, with meaningful autonomy and infrastructure access, while remaining corrigible and not exploiting reward loopholes. Better interpretability would help, but seeing a feature in a sparse autoencoder is not the same as understanding the system well enough to trust it. In the pessimistic direction, I would update sharply on clear evidence of strategic deception: a capable model recognizing evaluation conditions, concealing goals or abilities, and pursuing power or access when it expects not to be caught. Especially alarming would be AI-assisted AI research producing rapid capability gains before comparable safety progress. Timeline-wise, I would also change my view if skeptics identified a concrete bottleneck—compute, learning algorithms, data, reliability, or deployment—and made risky, quantitative predictions that kept succeeding. “Current models make silly mistakes” is weak evidence; “this specific limitation will prevent these specific capabilities through this date,” followed by repeated success, is much stronger. Likewise, consistently superhuman autonomous research would move me toward faster and larger impact. The discovery I want most is that alignment scales better than capability; the event I fear most is learning the opposite only after systems can hide it.
あなたはどの位置でしょうか?
いくつかの簡単な質問に答えて、自分のAIに対する世界観を探ってみましょう。
自分の世界観をマッピングする

あなたはどの位置でしょうか?

自分の世界観をマッピングする