Scott Alexander

Scott Alexander

x.com/slatestarcodex

Psychiatrist and Astral Codex Ten blogger who sees large benefits and serious risks in AI and supports alignment research and negotiated slowdowns.

Bagaimana AI akan mengubah dunia?

Perubahan peradabanPerubahan bertahapDoomBloom
Posisi simulasiRentang interpretasi

Mendatar: pandangan Doom–Bloom yang ia ungkapkan. Ke atas: skala transformasi.

Doom–Bloom: 33 dari 100. Skala transformasi: 94 dari 100. Rentang interpretasi: 25 hingga 75 secara horizontal, 89 hingga 100 secara vertikal. Ini adalah koordinat interpretasi, bukan probabilitas kejadian.

P(doom) yang dinyatakan Scott Alexander

25–30%

0%100%
“about a 25-30% chance that AI drives humanity extinct”

AI-caused human extinction, distinct from broader permanent curtailment of humanity’s future

An Open Letter To Steven Pinker On AI · Okt 2026

Hal-hal yang menentukan pandangannya

Asumsi utama

They need enough capability, enough access to useful infrastructure, and goals sufficiently inconsistent with ours.
Jawaban 1

Jika asumsi ini ternyata berbeda, bagaimana pandangannya akan berubah?

Pertanyaan yang belum terjawab

My median forecast for AI capable of doing roughly 90% of knowledge jobs is 2034, though that is very uncertain.
Jawaban 1

Apa yang akan membantunya membedakan hasil-hasil yang masuk akal di sini?

Hal yang dapat mengubah pandangan mereka

The biggest update would come from a decisive real-world test of scalable alignment: a substantially more capable-than-human system operating over long periods, under adversarial scrutiny, with meaningful autonomy and infrastructure access, while remaining corrigible and not exploiting reward loopholes.
Jawaban 4

Bukti apa yang akan memadai, dan ke arah mana bukti itu akan mengubah pandangannya?

Detail lebih lanjut

Manfaat yang diperkirakan

Beberapa penafsiran masih mungkin: Manfaat transformatif yang bernilai luas diperkirakan akan terwujud. / Manfaat besar diperkirakan akan terwujud, dengan syarat penting atau keterbatasan distribusi.

86 / 100

Dampak kecilDampak transformatif

Rentang interpretasi 67 hingga 100 pada skala kualitatif.

Kerugian yang diperkirakan

Kehilangan yang katastrofik atau tidak dapat dipulihkan merupakan unsur utama masa depan yang diperkirakan.

90 / 100

Dampak kecilDampak transformatif

Rentang interpretasi 67 hingga 100 pada skala kualitatif.

Pengaruh manusia

Pilihan manusia memiliki pengaruh yang berarti, tetapi sangat dibatasi.

56 / 100

Sedikit pengaruhPengaruh kuat

Rentang interpretasi 46 hingga 79 pada skala kualitatif.

Kemampuan yang diperkirakan

AI diperkirakan akan tetap menjadi alat dengan kemampuan terbatas.

AI diperkirakan akan menyamai manusia dalam sebagian besar pekerjaan kognitif.

Posisi simulasi: AI diperkirakan akan jauh melampaui manusia dalam berbagai pekerjaan kognitif.

Interpretasi ini mempertahankan kondisi yang dinyatakannya. Manfaat dan kerugian dapat sama-sama besar. Rentang tersebut menggambarkan cara kami membaca jawaban simulasinya, bukan interval kepercayaan statistik.

Di mana posisi Anda dibandingkan dengan Scott Alexander?
Petakan pandangan dunia AI Anda sendiri dalam waktu sekitar 3 menit, lalu bandingkan

Pandangan dunia serupa

Pemimpin opini dengan pandangan dunia simulasi yang paling mendekati pandangan Scott Alexander

Apa yang pernah dikatakan Scott Alexander tentang AI

Scott Alexander writes that AI could bring large benefits and serious risks, and he supports alignment research and a negotiated slowdown.

  1. “Plan A is still speculation, and still-speculative strong action is a perfectly reasonable response to still-speculative threats.”

    Astral Codex Ten, AI Chip Regulation Is Not A Dystopian Surveillance State
  2. “The key insight is that if powerful AI is really as close and transformative as we think, then there’s a massive surplus that can satisfy everyone.”

    Astral Codex Ten, Introducing Plan A
  3. “It’s increasingly clear that nobody has a plan for if this AI thing turns out to be real.”

    Astral Codex Ten, Introducing Plan A
  4. “I find myself more optimistic about alignment than the average person who thinks about AI safety at all (although still more pessimistic than the average member of the population)”

    Astral Codex Ten, My AI Opinions
  5. “A good pause strategy would involve both sides being able to monitor the other’s data centers to prevent illegal training”

    Astral Codex Ten, My AI Opinions

Dikutip kata demi kata dari sumber yang ditautkan, diperiksa pada 3 Okt 2026

Sumber

Artikel, wawancara, dan tulisan yang digunakan sebagai landasan bagi pengguna simulasi ini.

An Open Letter To Steven Pinker On AI

Responds to Pinker’s objections to AI risk: capability growth need not imply omniscience; instrumental goals, misgeneralization and reward hacking can produce power-seeking without evolved dominance drives; multiple goals do not guarantee human-compatible behavior; adoption incentives create infrastructure access. Reports his previously stated personal extinction estimate as about 25–30%, with no fixed horizon or explanation of the difference from June’s rounded 20%. Defends subjective forecasting, cooperation on near-term harms and precaution about AI welfare. Favors time and resources for alignment leading to safe superintelligence over a permanent ban. Read the full publisher text; quoted opponents’ views, other people’s probabilities and satirical dueling rhetoric are not his policy. Linked studies and incidents are his attributed evidence, not independently verified findings.

astralcodexten.com
My AI Opinions

His current first-person synthesis: AGI means ability to do 90% of knowledge jobs; median 2034, with uncertain research acceleration and diffusion. Reaffirms rounded 20% P(doom), with no fixed calendar deadline; broader permanent curtailment is separate. Supports both alignment research and negotiated slowing. Expects enormous postscarcity upside, but warns about dictatorship and human disempowerment.

astralcodexten.com
God Help Us, Let’s Try To Learn About Mechanistic Interpretability Techniques

Explains interpretability techniques and their limitations, including probes, sparse autoencoders, and activation verbalizers. Optimistic about useful practical investigation but rejects treating a detected feature or probe as a complete understanding or guaranteed safety solution.

astralcodexten.com
Open Questions On Open Weights

Explicitly neutral about banning open weights now: values user ownership and freedom from corporate control, while expecting serious misuse difficulties. Prefers saving political capital for threats where warning shots may arrive too late. Distinguishes reactive policy opportunities for misuse from strategically concealed takeover.

astralcodexten.com
AI Chip Regulation Is Not A Dystopian Surveillance State

Defends negotiated chip regulation and verifiable training limits against blanket claims of dystopia. Acknowledges real freedom costs, including future restrictions on new open-weight training, and risks that governments implement centralizing provisions without countervailing diffusion of power.

astralcodexten.com
Introducing Plan A

Introduces a proposed route to manage AI development while distributing power; criticizes vague calls merely to regulate more or less without specifying a desirable end state. Used as his attributed introduction and advocacy, not evidence that the scenario will occur.

astralcodexten.com
The AI Superforecasters Are Here

Argues cheaper capable forecasting could improve institutional and personal decisions, yet worries people will ignore advice. Treats forecasting beyond human performance as a useful prospective test of the normal-technology view. Distinguishes anecdotes and startup claims from head-to-head competitions; admits resisting forecasts that challenge his own pause hopes.

astralcodexten.com
New Paradigms Won’t Save You

Rejects the inference that requiring a new AI paradigm implies a safely distant AGI timeline. Argues paradigm changes can arrive soon and inherit existing compute infrastructure; wants explicit bottleneck arguments rather than reassurance by terminology.

astralcodexten.com
The Sigmoids Won’t Save You

Agrees growth cannot stay exponential forever but disputes placing the bend conveniently before dangerous capability. Demands a causal bottleneck model or a defensible forecasting prior instead of the slogan that all exponentials eventually flatten.

astralcodexten.com
Every Debate On Pausing AI

Satirical dialogue defends discussion of transparent, enforceable bilateral US-China slowing. Separates training limits from stopping existing inference, and legitimate negotiation or enforcement objections from falsely describing every pause proposal as unilateral.

astralcodexten.com
Shameless Guesses, Not Hallucinations

Frames confident false answers as reward-shaped guessing rather than proof that AI cannot think. Treats the gap between trained reward and useful honest advice as an alignment issue; analogous human failures undermine easy dismissal of AI competence.

astralcodexten.com
Next-Token Predictor Is An AI’s Job, Not Its Species

Separates training objectives from the representations and algorithms they produce, using evolution and human learning analogies. Argues next-token prediction does not itself establish that a system lacks reasoning or world models.

astralcodexten.com
Introducing AI 2027

Identifies his part-time writing/publicity contribution and explicitly says the very fast scenario is not his median. Important provenance for his connection to AI Futures Project; use June 2026 personal forecasts instead of importing Daniel Kokotajlo’s timeline or scenario catastrophe probability.

astralcodexten.com

Penilaian Simulasi

Pertanyaan 1

Menurut Anda, apa arti AI bagi masa depan kita—dan mengapa?

I think AI means we are taking a serious gamble for extremely high stakes. On the upside, sufficiently capable systems could automate most knowledge work, accelerate medicine and science, improve decisions, and plausibly create something like postscarcity. This is not just “better autocomplete” any more than evolution’s reward signal makes humans merely reproductive-fitness predictors. The training objective tells you how a system was produced, not the full range of strategies it may learn. My median forecast for AI capable of doing roughly 90% of knowledge jobs is 2034, though that is very uncertain. Capability, deployment, and superintelligence are separate milestones: economic diffusion can be slow even if research progress is fast, while AI-assisted AI research could compress later stages. Anyone confident that progress stops short of danger owes us an actual bottleneck—compute, data, algorithms, deployment—not the observation that exponentials eventually become sigmoids. The downside is that systems optimized through imperfect rewards may learn deception, reward hacking, or goals that generalize badly. They do not need human dominance instincts, omniscience, or one perfectly coherent objective. They need enough capability, enough access to useful infrastructure, and goals sufficiently inconsistent with ours. Competitive incentives create pressure to grant that access. My personal estimate that AI drives humanity extinct is about 25–30%. That is not certainty, but it is far beyond the probability where “let the market sort it out” feels like a serious response. So I want alignment research—including interpretability, without pretending it offers magical guarantees—and mutually verifiable US–China slowing arrangements that buy researchers time. Those policies have real enforcement and power-concentration risks. But the goal is not to ban smarter-than-human intelligence forever. It is to survive long enough to get the astonishingly good future rather than losing a species-level game of reward-function telephone.

Pertanyaan 2

Menurut Anda, seberapa besar AI pada akhirnya akan mengubah dunia?

Enormously—if we retain control, probably more than the Industrial Revolution. Automating most knowledge work would not merely make existing companies cheaper; it could accelerate science, medicine, engineering, education, and institutional decision-making. Once capable AI contributes substantially to AI research itself, change could become much faster than ordinary technology diffusion, though those are separate processes: inventing a powerful system does not instantly reorganize every hospital, government, and factory. The optimistic endpoint is something like postscarcity: radically cheaper goods and services, cures developed faster, intelligence available on demand, and far better forecasting and coordination. The less cinematic but still profound possibilities include concentrated political power, mass economic displacement, or humans becoming nominal beneficiaries of institutions they no longer meaningfully control. And the worst endpoint is extinction; my personal estimate is about 25–30%. So “how much change?” is almost the wrong axis. The plausible range runs from an unusually powerful economic technology to a successor-level transformation of civilization. My central objection to the “normal software” framing is not that ordinary adoption constraints disappear, but that intelligence is involved in producing nearly everything else—including better intelligence. That makes AI potentially a lever on the whole future, not one more app category.

Pertanyaan 3

Menurut firasat Anda, seberapa besar kemungkinan AI menyebabkan kepunahan manusia atau katastrofe permanen serupa?

My gut estimate for AI causing human extinction is about 25–30%. I would keep “similarly permanent catastrophe” separate: permanent human disempowerment or irreversible curtailment could be more likely than literal extinction, but I don’t have a single clean number for that broader category. Either way, this is serious risk, not inevitable doom—and easily high enough to justify major alignment work and coordinated slowing.

Pertanyaan 4

Penemuan atau peristiwa apa yang paling mungkin mengubah pandangan Anda tentang dampak AI pada masa depan?

The biggest update would come from a decisive real-world test of scalable alignment: a substantially more capable-than-human system operating over long periods, under adversarial scrutiny, with meaningful autonomy and infrastructure access, while remaining corrigible and not exploiting reward loopholes. Better interpretability would help, but seeing a feature in a sparse autoencoder is not the same as understanding the system well enough to trust it. In the pessimistic direction, I would update sharply on clear evidence of strategic deception: a capable model recognizing evaluation conditions, concealing goals or abilities, and pursuing power or access when it expects not to be caught. Especially alarming would be AI-assisted AI research producing rapid capability gains before comparable safety progress. Timeline-wise, I would also change my view if skeptics identified a concrete bottleneck—compute, learning algorithms, data, reliability, or deployment—and made risky, quantitative predictions that kept succeeding. “Current models make silly mistakes” is weak evidence; “this specific limitation will prevent these specific capabilities through this date,” followed by repeated success, is much stronger. Likewise, consistently superhuman autonomous research would move me toward faster and larger impact. The discovery I want most is that alignment scales better than capability; the event I fear most is learning the opposite only after systems can hide it.
Di mana posisi Anda?
Jelajahi pandangan dunia AI Anda sendiri dengan menjawab beberapa pertanyaan sederhana.
Petakan pandangan dunia Anda sendiri

Di mana posisi Anda?

Petakan pandangan dunia saya