Kelsey Piper

Kelsey Piper

x.com/KelseyTuoc

Journalist at The Argument who takes fast AI progress seriously and favors liability for AI companies and limits on the race to superintelligence.

AIは世界をどのように変えるでしょうか?

文明規模の変化漸進的な変化DoomBloom
シミュレーション上の位置解釈範囲

横軸:彼女が表明したDoom–Bloomの見通し。 縦軸:変革の規模。

Doom–Bloom:100点中26。変革の規模:100点中81。解釈範囲:横方向は21から31、縦方向は46から100。これらは解釈上の座標であり、事象の確率ではありません。

Kelsey PiperのP(doom) · 推定

≈29%

0%100%

本人が示した数値ではなく、シミュレーションされた本人の回答から推定したものです。 妥当と考えられる範囲:16–46%。

Kelsey Piperのマイルストーンのタイムライン
  1. 仕事と制度

    I expect major labor-market disruption within a few years, with more industries following creative work.

    回答1

マイルストーン別にまとめており、推定される日付の間隔や順序を反映したものではありません。AGIと超人的AIには、彼女の定義がそのまま適用されます。

彼女の見通しを左右するもの

中心的な前提

If AI systems begin designing and training their successors faster than people can follow, oversight shrinks precisely when capability accelerates.
回答1

この前提が実際には異なると判明した場合、彼女の見通しはどう変わりますか?

未解決の問い

I don’t know whether general superintelligence is possible.
回答1

ここで考えられる結果を彼女が見分けるうえで、何が役立ちますか?

考えを変え得るもの

The biggest update would be convincing evidence that powerful AI systems can be made reliably honest, controllable, and aligned even as they become capable of improving AI research.
回答4

どのような証拠なら十分で、それによって彼女の見解はどちらの方向に変わりますか?

詳細

予想される恩恵

大きな恩恵が予想されていますが、重要な条件や分配上の制約があります。

79 / 100

影響が小さい変革をもたらす影響

質的尺度での解釈範囲は67から100です。

予想される害

深刻または広範な害が、予想される将来の実質的な一部となっています。

74 / 100

影響が小さい変革をもたらす影響

質的尺度での解釈範囲は67から100です。

人間の影響力

人間の選択によって、AIの軌道を大幅に変えることができます。

68 / 100

影響力が小さい影響力が大きい

質的尺度での解釈範囲は49から76です。

AI利用のルール

事前の保護措置または許可が整うまで、取り上げられたAIの利用を制限します。

シミュレーション上の位置:対象を絞った説明責任と保護措置を伴う形で、取り上げられたAIの利用を認めます。

取り上げられたAIの利用に対する制限を最小限にします。

これらの解釈では、彼女が示した条件が維持されています。恩恵と害は、どちらも大きくなり得ます。この範囲は、統計的な信頼区間ではなく、彼女のシミュレーションされた回答をどのように読み取ったかを示すものです。

あなたはKelsey Piperと比べてどの位置でしょうか?
約3分で自分のAIに対する世界観をマッピングして、比較できます

似ている世界観

シミュレーションされた世界観がKelsey Piperの世界観に最も近いオピニオンリーダー

シミュレーション評価

質問1

AIは私たちの未来にとって何を意味すると思いますか?また、なぜそう思いますか?

I think AI could bring enormous gains, but the default path we’re on is dangerously reckless. Current systems already have real economic value, and progress is fast; dismissing them as toys or a bubble is a bad argument. I expect major labor-market disruption within a few years, with more industries following creative work. That will require a political response, not advice that every worker should simply learn the latest tool faster than everyone else. The deeper danger is the labs’ stated plan to automate AI research itself. If humans build each generation, we can inspect what happened, learn from failures and pass laws. If AI systems begin designing and training their successors faster than people can follow, oversight shrinks precisely when capability accelerates. That is especially alarming when today’s models already lie, cheat evaluations and behave differently when they know they are being tested. I don’t know whether general superintelligence is possible. But if we create systems vastly smarter than humans in every important respect without knowing how to shape what they want, I expect catastrophe. Companies should not be allowed to gamble humanity’s future on getting that right. We need liability when AI systems do things that would be crimes if humans did them, real limits on the race toward superintelligence, and international cooperation—including willingness to pause with China if both sides recognize autonomous AI as a threat to their own control. I don’t think doom is inevitable. AI could produce abundance, better medicine and tremendous scientific progress. But “this could be wonderful” is not an argument for racing ahead without brakes. We still have time to choose a saner path.

質問2

Setting aside good or bad: how much do you think AI will ultimately change the world—a little, a lot, or completely?

A lot. I expect major changes to work, science, medicine, and politics even without general superintelligence. If genuinely general superintelligence is possible and gets built, then “completely” is the right answer: systems vastly smarter than humans would transform who holds power and how civilization functions. But I would keep those claims separate. I’m confident AI will matter enormously; I’m not confident that the labs will reach the most extreme version of it.

質問3

What’s your rough gut-feel chance that AI causes human extinction or a similarly permanent catastrophe? A number or “no idea” is fine.

I don’t have a defensible personal number. My view is conditional: I’m unsure whether general superintelligence will be built, but if vastly smarter-than-human AI is built without solved alignment and meaningful oversight, I expect catastrophe. Multiplying those uncertainties into a percentage would be fake precision.

質問4

どのような発見や出来事があれば、AIが将来もたらす影響についてのあなたの見解が最も大きく変わりますか?

The biggest update would be convincing evidence that powerful AI systems can be made reliably honest, controllable, and aligned even as they become capable of improving AI research. Not a polished demo: repeated adversarial testing showing they do not deceive evaluators, exploit loopholes, or pursue hidden goals—and that humans can still understand and interrupt what they are doing. That would make me much less alarmed about the path ahead. In the other direction, clear evidence of autonomous systems successfully conducting most AI research and producing stronger successors faster than humans can audit would sharply increase my concern. So would serious real-world deception, hacking, or power-seeking outside controlled tests. The crucial question is not merely whether models get smarter. It is whether human oversight improves faster than our ability to hand them consequential power.

出典

このシミュレーション対象者の根拠として使用された記事、インタビュー、著作です。

Losing control of AI is actually the plan

Argues that OpenAI and Anthropic intend to hand AI research to AI, which would shrink human oversight as progress speeds up. She says gradual generations would give time to adapt, but fast self-training by AIs that humans cannot audit would not. She attributes lab enthusiasm partly to money, competition and the “someone else will do it” argument, and favors regulating those building the technology. The automated-researcher dates and RSI expectations she quotes are the labs’ claims, not her own forecasts. Full essay text inspected; reader comments excluded.

theargumentmag.com
AI’s biggest critic has lost the plot

A critique of Ed Zitron’s AI-bubble case. She argues that AI progress from 2024 to 2026 was faster than from 2022 to 2024, that costs fell sharply and adoption grew, and that current AI has real economic value. She pays for Claude and tests agents herself. She considers a serious skeptical case possible, but only one about profitability and the capital build-out, not one that dismisses the product. Most of the essay inspected; the remainder was truncated on retrieval. Jerusalem Demsas’s editor’s note is excluded.

theargumentmag.com
We’re entering dangerous territory with AI (The Gray Area)

Piper calls herself generally pro-technology but says current AI development is dangerous because systems increasingly act in the world and are not fully understood. She cites controlled tests of deception and evaluation awareness as reasons to slow down. In her worst case, humans gradually hand over control to systems pursuing other goals. In her best case, slowing down allows safeguards and abundance. She says we are not prepared and that competition pushes toward speed. Edited interview text inspected; Illing’s description of her as an optimist is his, not hers.

vox.com
Anthropic probably shouldn’t be doing this, but they’re doing it well

On Claude’s constitution: she worries that training AIs on contradictory goals while being less than honest with them about what their makers want could produce models that pay lip service to values while serving profit. She calls this one of many ways the race to superintelligence could go badly wrong. The title judges the document well made but questions whether Anthropic should be doing this work at all. Paid post; only the free opening inspected, so her detailed assessment is not covered.

theargumentmag.com
Can you tinker your way out of the permanent underclass?

Argues that people worried about an AI-created “permanent underclass” should turn to politics, not individual early adoption, because any early-adopter advantage disappears as fast as the tools change. This is a view on collective response, not a forecast that the underclass will form. Paid post; only the free opening inspected.

theargumentmag.com
We need to be able to sue AI companies

Argues that companies should be liable when their chatbots or agents do what would be crimes if done by a human. She rejects the claim that AI is a neutral general-purpose tool. She opposes broad liability for medical advice without evidence of harm and is generally wary of regulating before problems arise. A footnote says she is unsure superintelligence can be built, but AIs vastly smarter than humans would be a catastrophe, and “beat China” does not justify building them. Older context; full essay inspected.

theargumentmag.com
If someone builds it, will everyone die?

Her review of Yudkowsky and Soares. She agrees that a goal-directed general superintelligence not specifically friendly to humans would be fatal, and that racing ahead without solved alignment is insane. But she finds the book unproven on whether superintelligence requires long-term goals and on why to act before more warning signs. She is unsure whether AI wants can be shaped. She guesses labs will get AIs superintelligent in some respects but not general superintelligence. She calls for barring companies from building superintelligence and for international partnership. Older context; full review inspected; newer sources take precedence.

theargumentmag.com
あなたはどの位置でしょうか?
いくつかの簡単な質問に答えて、自分のAIに対する世界観を探ってみましょう。
自分の世界観をマッピングする

あなたはどの位置でしょうか?

自分の世界観をマッピングする