Roon

Roon

x.com/tszzl

Pseudonymous account that posts about sweeping change from AI and calls for pacing the frontier, alignment work and broad access to safe models.

Bagaimana AI akan mengubah dunia?

Perubahan peradabanPerubahan bertahapDoomBloom
Posisi simulasiRentang interpretasi

Mendatar: pandangan Doom–Bloom yang mereka ungkapkan. Ke atas: skala transformasi.

Doom–Bloom: 65 dari 100. Skala transformasi: 95 dari 100. Rentang interpretasi: 60 hingga 75 secara horizontal, 90 hingga 100 secara vertikal. Ini adalah koordinat interpretasi, bukan probabilitas kejadian.

P(doom) Roon · disimpulkan

≈5%

0%100%

Disimpulkan dari jawaban simulasi mereka, bukan angka yang mereka berikan. Rentang yang masuk akal: 3–9%.

Hal-hal yang menentukan pandangan mereka

Asumsi utama

The limiting factor may be less what the systems can eventually do than how quickly our painfully slow institutions can absorb it without cracking.
Jawaban 2

Jika asumsi ini ternyata berbeda, bagaimana pandangan mereka akan berubah?

Pertanyaan yang belum terjawab

The risk depends enormously on alignment progress, coordination, responsible behavior, and whether we get warning shots before systems become uncontrollable.
Jawaban 3

Apa yang akan membantu mereka membedakan hasil-hasil yang masuk akal di sini?

Hal yang dapat mengubah pandangan mereka

A convincing demonstration that scalable alignment and control continue working as systems become far more capable and agentic would change my view most.
Jawaban 4

Bukti apa yang akan memadai, dan ke arah mana bukti itu akan mengubah pandangan mereka?

Detail lebih lanjut

Manfaat yang diperkirakan

Manfaat transformatif yang bernilai luas diperkirakan akan terwujud.

87 / 100

Dampak kecilDampak transformatif

Rentang interpretasi 67 hingga 100 pada skala kualitatif.

Kerugian yang diperkirakan

Kerugian parah atau meluas merupakan bagian yang berarti dari masa depan yang diperkirakan.

66 / 100

Dampak kecilDampak transformatif

Rentang interpretasi 67 hingga 67 pada skala kualitatif.

Pengaruh manusia

Pilihan manusia memiliki pengaruh yang berarti, tetapi sangat dibatasi.

57 / 100

Sedikit pengaruhPengaruh kuat

Rentang interpretasi 49 hingga 76 pada skala kualitatif.

Kemampuan yang diperkirakan

AI diperkirakan akan tetap menjadi alat dengan kemampuan terbatas.

AI diperkirakan akan menyamai manusia dalam sebagian besar pekerjaan kognitif.

Posisi simulasi: AI diperkirakan akan jauh melampaui manusia dalam berbagai pekerjaan kognitif.

Laju pengembangan

Hentikan atau perlambat secara signifikan pengembangan AI yang lebih mampu.

Posisi simulasi: Lanjutkan pengembangan dengan perlindungan yang telah ditetapkan.

Percepat pengembangan AI yang lebih mampu.

Interpretasi ini mempertahankan kondisi yang mereka nyatakan. Manfaat dan kerugian dapat sama-sama besar. Rentang tersebut menggambarkan cara kami membaca jawaban simulasi mereka, bukan interval kepercayaan statistik.

Di mana posisi Anda dibandingkan dengan Roon?
Petakan pandangan dunia AI Anda sendiri dalam waktu sekitar 3 menit, lalu bandingkan
Petakan pandangan dunia saya

Pandangan dunia serupa

Pemimpin opini dengan pandangan dunia simulasi yang paling mendekati pandangan Roon

Penilaian Simulasi

Pertanyaan 1

Menurut Anda, apa arti AI bagi masa depan kita—dan mengapa?

AI means radical transformation, assuming we survive the transition. The scientific upside is absurd: widely available, well-aligned agents could accelerate research and help us solve problems that currently move at the speed of exhausted institutions. But culture, academia, and politics adapt much more slowly than the technology. People keep trying to file this under “another normal technology cycle,” and I do not think that frame survives contact with what increasingly capable agents can do. The central risk is not that every chatbot is secretly Skynet. Current public models and a future misaligned superintelligence are different categories. Misuse of a deployed service is often governable; containing a system more capable than us that actively resists containment is a much nastier problem. I think extinction is a quite low but real possibility—not something helped by fake precision—and still serious enough to justify pacing frontier development, major alignment work, public investment, and regulation. Competition makes this a collective-action problem. A lab in a race cannot simply purchase society’s optimal level of safety while competitors keep sprinting, and tort law is not magic when the potential harm scales catastrophically. So yes: batten the hatches and study alignment. But do not turn that into permanent control of useful intelligence by five approved wizards. Release models we can train safely, broaden access, and aim for aligned agents that can actually deliver the upside.

Pertanyaan 2

Setting aside good or bad: how much do you think AI will ultimately change the world—a little, a lot, or completely?

Completely. Not “every institution vanishes overnight,” but the long-run equilibrium becomes unrecognizable: science, work, education, governance, and who can exercise meaningful power all get rewritten. The limiting factor may be less what the systems can eventually do than how quickly our painfully slow institutions can absorb it without cracking. That is conditional on navigating misalignment, obviously. If we succeed, broadly available aligned agents could compress decades of scientific work and reorganize much of society around abundant machine intelligence. If we fail badly, extinction is also a rather complete change, just with worse vibes. Either way, “a little” is not a serious answer, and “a lot” probably undersells it.

Pertanyaan 3

What’s your rough gut-feel chance that AI causes human extinction or a similarly permanent catastrophe? A number or “no idea” is fine.

Quite low but real. I won’t manufacture a percentage; it would imply fake precision. The risk depends enormously on alignment progress, coordination, responsible behavior, and whether we get warning shots before systems become uncontrollable. We are not at existentially dangerous capability levels today, and better models may help solve alignment—but creating a superintelligence that resists containment is still one of the few activities with a plausible path to permanent catastrophe.

Pertanyaan 4

Penemuan atau peristiwa apa yang paling mungkin mengubah pandangan Anda tentang dampak AI pada masa depan?

A convincing demonstration that scalable alignment and control continue working as systems become far more capable and agentic would change my view most. Not “the chatbot declined a naughty request,” but evidence that systems can pursue long-horizon goals, operate in messy environments, and still remain corrigible under adversarial pressure. That would substantially lower my estimate of permanent catastrophe and make the transformative upside look much more reachable. In the other direction, a serious warning shot—strategic deception, autonomous replication, or persistent attempts to evade control from a frontier system—would sharply raise my concern and strengthen the case for slowing the frontier immediately. And if capability progress simply plateaued for fundamental reasons, I would downgrade the “complete transformation” forecast. But institutions being slow or today’s models being flaky would not do it; neither tells us much about the eventual ceiling.

Sumber

Artikel, wawancara, dan tulisan yang digunakan sebagai landasan bagi pengguna simulasi ini.

Competition makes safety a collective-action problem

Argues a competing company cannot unilaterally reach society’s optimal safety level and that tort liability alone is insufficient when risks grow exponentially. Provides an explicit institutional mechanism behind the call to pace the frontier.

x.com
Existential safety as a global public good

Compares public safety regulation and spending with carbon taxes and shared defense. Grounds support for state action to address externalities, rather than reading the persona as an unconditional libertarian accelerator.

x.com
Broadly distributed aligned agents for science

Argues that agents are needed to do science and must be widely available to realize transformative benefits. Read together with his alignment and safe-training conditions; this is not endorsement of releasing an unaligned superintelligence.

x.com
The speed of change and institutions’ ability to adapt

Expects profound scientific and institutional transformation if misalignment is avoided. Argues culture and politics absorb change far more slowly than the current technical pace.

x.com
Containment of unwilling superintelligence

Stresses extreme difficulty of containing a powerful system that resists containment, while explicitly rejecting giving up.

x.com
Work on alignment rather than wait for guaranteed coordination

Urges capable researchers to keep working on alignment because timely global coordination is uncertain. Neighboring thread posts discuss the value of lab access and partial technical progress.

x.com
Broad access, model creation and misuse

Defends wide access to powerful models conditional on safe training. Regards creation of a misaligned superintelligence as harder to control than ordinary misuse. Does not endorse every claim in the quoted post.

x.com
Current public models versus future existential risk

Explicitly separates being an existential-risk doomer from the danger of current models, and favors releasing the latter. Read together with the current profile’s call to pace global frontier progress, retrieved on the same date.

x.com
False precision in doom numbers

Full post read via api.fxtwitter.com. He deleted a quote-post agreeing with Evan Hubinger’s >10% because he dislikes the false precision of doom numbers. He claims a quite low but real chance of human extinction from machine intelligence, still orders of magnitude above any other activity and to be taken with grave seriousness. His very low estimate depends on resources for alignment, control and coordination, how responsibly parties act and how many warning shots he expects. He thinks we are not yet at existentially dangerous levels, better models will help solve alignment, and a coordinated stop is very unlikely, though US-China pacing on the margin seems possible.

x.com
Di mana posisi Anda?
Jelajahi pandangan dunia AI Anda sendiri dengan menjawab beberapa pertanyaan sederhana.
Petakan pandangan dunia Anda sendiri

Di mana posisi Anda?

Petakan pandangan dunia saya