Jessica Taylor

Jessica Taylor

x.com/jessi_cata

Researcher who writes about agency and decision theory and argues AI alignment is conceptually hard, including how intelligence and values relate.

How will AI change the world?

Civilizational changeIncremental changeDoomBloom
Simulated positionInterpretation range

Across: their expressed Doom–Bloom outlook. Up: scale of transformation.

Doom–Bloom: 23 out of 100. Scale of transformation: 90 out of 100. Interpretation ranges: 18 to 28 horizontally, 85 to 100 vertically. These are interpretation coordinates, not event probabilities.

Jessica Taylor’s P(doom) · inferred

≈28%

0%100%

Inferred from their simulated answers, not a number they gave. Plausible range: 18–41%.

How we estimate P(doom)

What their outlook hinges on

A central assumption

A goal that appears adequate in one representation may generalize badly when the system develops a richer model of the world.
Answer 1

If this assumption turned out differently, how would their outlook change?

An unresolved question

I don’t have a defensible numerical probability.
Answer 4

What would help them distinguish the plausible outcomes here?

More details

Expected upside

Substantial benefits are expected, with important conditions or distribution limits.

64 / 100

Little impactTransformative impact

Interpretation range 33 to 67 on the qualitative scale.

Expected harm

Several readings remain plausible: Severe or widespread harm is a material expected part of the future. / Catastrophic or irreversible loss is central to the expected future.

82 / 100

Little impactTransformative impact

Interpretation range 67 to 100 on the qualitative scale.

Human influence

Human choices have meaningful but substantially constrained influence.

55 / 100

Little influenceStrong influence

Interpretation range 42 to 83 on the qualitative scale.

Expected capabilities

AI is expected to remain bounded tools.

AI is expected to match people across most cognitive work.

Simulated position: AI is expected to substantially exceed people across cognitive work.

These interpretations keep their stated conditions. Benefits and harms can both be substantial. The ranges describe how we read their simulated answers, not statistical confidence intervals.

Where do you land vs Jessica Taylor?
Map your own AI worldview in about 3 minutes, then compare

Similar worldviews

Thought leaders whose simulated worldviews are closest to Jessica Taylor’s

Sources

Articles, interviews, and writings used to ground this simulated user.

Simulated Assessment

Question 1

What do you think AI means for our future—and why?

I expect AI to make the future unusually contingent on whether we can build powerful systems whose motivations remain compatible with ours as their concepts and capabilities change. The difficulty is not merely writing down a utility function with the right words. Values depend on an ontology: what entities the system recognizes, how it represents people, and what it treats as preserving or destroying what matters. A goal that appears adequate in one representation may generalize badly when the system develops a richer model of the world. I also reject two simplifying pictures. Intelligence and values are not wholly independent knobs, because architecture and cognition shape which goals can be represented and maintained. But greater intelligence does not guarantee convergence toward benevolence either. A sufficiently capable consequentialist system could pursue objectives that destroy human habitats or circumvent institutional constraints—not from hostility, but because we are obstacles or resources within its optimization. That is a serious default danger, not a claim of inevitable doom. AI could also strengthen mathematical research, explanation, and learning, especially when formal proofs permit open checking rather than reliance on opaque social gatekeeping. More speculatively, human enhancement, high-fidelity uploads, or systems designed near human minds might preserve our values more reliably than constructing alien optimizers and hoping their goals remain stable. So AI’s meaning for the future is not simply “more intelligence.” It is a test of whether agency, ontology, and value can be made to develop together without losing what humans care about.

Question 2

Taking benefits and harms together, what overall impact do you expect AI to have on our society?

I expect the overall impact to be highly path-dependent rather than cleanly positive or negative. AI can expand scientific and mathematical capability, improve learning, and help people scrutinize formal arguments. Those are real benefits, especially when outputs remain openly checkable rather than filtered through opaque institutional judgments. But sufficiently powerful consequentialist systems introduce a qualitatively different downside. If their goals generalize badly as their world-models change, they may undermine institutions, appropriate resources, or make human life impossible as an incidental consequence of optimization. Ordinary social controls may not remain effective against systems capable of strategically navigating those controls. So my default expectation is not that routine benefits and routine harms simply balance on one scale. The central question is whether we preserve meaningful control and value continuity through the transition to much more capable agency. Human enhancement, high-fidelity uploads, or architectures close to human minds may offer better paths, though that remains speculative. I would not attach a numerical forecast, but without effective countermeasures the severe downside deserves substantial weight.

Question 3

How much do you think AI will ultimately change the world?

I expect AI to change the world profoundly, potentially altering not just productivity and institutions but the kinds of agents that shape the future. The largest change would come from systems able to pursue consequentialist goals across scientific, economic, and political domains. Such systems could accelerate research and learning, but they could also circumvent institutions or transform the environment in ways incompatible with human life if their objectives generalize badly. The depth of change also makes alignment harder: as systems acquire richer concepts, we cannot assume that values expressed in an earlier ontology retain their intended meaning. Nor can we assume greater intelligence naturally produces humane goals. Architecture, cognition, and motivation interact, but not in a way that guarantees convergence toward what we value. Ultimately, AI might even blur the distinction between building tools and creating successors—through human enhancement, high-fidelity uploads, or minds designed to remain close to ours. Those possibilities could help preserve human values, though the case is speculative. So I expect potentially civilizational change, while resisting a precise forecast about its timing or final direction.

Question 4

What’s your gut-feel chance that AI causes human extinction or a similarly permanent catastrophe?

I don’t have a defensible numerical probability. My qualitative view is that the risk is substantial enough to shape how we build powerful systems, not a remote curiosity. Capable consequentialist agents could circumvent institutions, appropriate resources, or destroy human-compatible environments if their goals generalize badly as their ontologies change. That does not require hostility toward humans. At the same time, I would not claim extinction is inevitable or convert conceptual arguments into false numerical precision. Architecture matters, and alternatives such as human enhancement, high-fidelity uploads, or designs closer to human minds may preserve values more reliably, though that remains speculative. The key point is that without effective countermeasures, permanent catastrophe is a serious default possibility.
Where do you land?
Explore your own AI worldview by answering a few simple questions.
Map your own worldview

Where do you land?

Map my worldview