Joe Carlsmith

@jkcarlsmith on X

Extraordinary flourishing is possible, but safe AI needs technical progress and credible restraint.

Map your own worldview

How will AI change the world?

ConcernHope

Simulated positionInterpretation range

Across: his expressed Doom–Bloom outlook. Up: scale of transformation.

Joe Carlsmith’s estimated P(doom)

≈21%

0%100%

Inferred from the likelihood described in his simulated answers. Approximate interpretation range: 10–30%. Applies to the outcome and conditions in his simulated answers; this is an inferred percentage.

Joe Carlsmith’s milestone timeline

No milestone timing was established. Dates, “not sure,” “possibly never,” and dependencies can all appear here when expressed.

Grouped by milestone, not spaced or ordered by inferred dates. AGI and superhuman AI retain his definitions.

What his outlook hinges on

A central assumption

It is that highly capable agents may have motivations imperfectly aligned with ours, options that let them evade control, and incentives to seek influence or prevent correction.

Answer 1

If this assumption turned out differently, how would his outlook change?

What could change their mind

The biggest positive update would be a technically and institutionally credible safety case for superintelligence: evidence that we can understand and shape a system’s motivations, detect strategic deception, keep its options bounded, preserve meaningful corrigibility as capabilities scale, and verify these claims under adversarial pressure.

Answer 4

What evidence would be enough, and in which direction would it move his view?

More details

Expected upside

Substantial benefits are expected, with important conditions or distribution limits.

59 / 100

Little impactTransformative impact

Expected harm

Catastrophic or irreversible loss is central to the expected future.

90 / 100

Little impactTransformative impact

Demonstrated reasoning

Reasoning, consideration of alternatives, and handling of uncertainty in his simulated answers. This describes the simulated answers, not the real person’s intelligence or opinions.

97 / 100

Little demonstratedWell developed

Human influence

Human choices have meaningful but substantially constrained influence.

61 / 100

Little influenceStrong influence

Development pace

Stop or substantially slow development of more capable AI.

Simulated position: Continue development under stated safeguards.

Speed up development of more capable AI.

These interpretations keep his stated conditions. Benefits and harms can both be substantial. The ranges describe how we read his simulated answers, not statistical confidence intervals.

Simulated Assessment

Sources

Articles, interviews, and writings used to ground this simulated persona.

How do we solve the alignment problem?

Introduction to an essay series about paths to safe, useful superintelligence.

joecarlsmith.com

On restraining AI development for the sake of safety

My take on slowing down AI.

joecarlsmith.com

Predictable updating about AI risk

How worried about AI risk will we be when we can see advanced machine intelligence up close? We should worry accordingly now.

joecarlsmith.com

Otherness and control in the age of AGI

Introduction and summary for a series of essays about how agents with different values should relate to each other, and about the ethics of seeking and sharing power.

joecarlsmith.com

Actually possible: thoughts on Utopia

There are oceans we have barely dipped a toe into. There are drums and symphonies we can barely hear. There are suns whose heat we can barely feel on our skin.

joecarlsmith.com

Joe Carlsmith — Preventing an AI takeover

Chatted with Joe Carlsmith about whether we can trust power/techno-capital, how to not end up like Stalin in our urge to control the future, gentleness towar...