Jessica Taylor

Jessica Taylor

x.com/jessi_cata

Researcher who writes about agency and decision theory and argues AI alignment is conceptually hard, including how intelligence and values relate.

Comment l’IA changera-t-elle le monde ?

Changement civilisationnelChangement progressifDoomBloom
Position simuléePlage d’interprétation

Horizontalement : leur perspective Doom–Bloom telle qu’elle a été exprimée. Verticalement : ampleur de la transformation.

Doom–Bloom : 23 sur 100. Ampleur de la transformation : 90 sur 100. Plages d’interprétation : de 18 à 28 horizontalement, de 85 à 100 verticalement. Il s’agit de coordonnées d’interprétation, et non de probabilités d’événements.

P(doom) de Jessica Taylor · inféré

≈28%

0%100%

Déduit de leurs réponses simulées, et non d’un chiffre donné par ces personnes. Plage plausible : 18–41%.

Comment nous estimons P(doom)

Ce dont dépend leur perspective

Une hypothèse centrale

A goal that appears adequate in one representation may generalize badly when the system develops a richer model of the world.
Réponse 1

Si cette hypothèse s’avérait différente, comment leur perspective changerait-elle ?

Une question non résolue

I don’t have a defensible numerical probability.
Réponse 4

Qu’est-ce qui les aiderait à distinguer les résultats plausibles ici ?

Plus de détails

Bénéfices attendus

Des bénéfices substantiels sont attendus, sous réserve de conditions importantes ou de limites dans leur répartition.

64 / 100

Faible impactImpact transformateur

Plage d’interprétation de 33 à 67 sur l’échelle qualitative.

Dommages attendus

Plusieurs interprétations restent plausibles : Des dommages graves ou généralisés constituent une composante substantielle de l’avenir attendu. / Une perte catastrophique ou irréversible occupe une place centrale dans l’avenir attendu.

82 / 100

Faible impactImpact transformateur

Plage d’interprétation de 67 à 100 sur l’échelle qualitative.

Influence humaine

Les choix humains ont une influence significative, mais fortement contrainte.

55 / 100

Faible influenceForte influence

Plage d’interprétation de 42 à 83 sur l’échelle qualitative.

Capacités attendues

L’IA devrait rester un ensemble d’outils aux capacités limitées.

L’IA devrait égaler les humains dans la plupart des tâches cognitives.

Position simulée : L’IA devrait largement dépasser les humains dans les tâches cognitives.

Ces interprétations conservent les conditions énoncées. Les bénéfices et les dommages peuvent tous deux être substantiels. Les plages décrivent notre lecture de leurs réponses simulées, et non des intervalles de confiance statistiques.

Où vous situez-vous par rapport à Jessica Taylor ?
Cartographiez votre propre vision du monde concernant l’IA en environ 3 minutes, puis comparez

Visions du monde similaires

Leaders d’opinion dont les visions du monde simulées sont les plus proches de celle de Jessica Taylor

Sources

Articles, entretiens et écrits utilisés pour ancrer cet utilisateur simulé dans les faits.

Évaluation simulée

Question 1

Selon vous, que signifie l’IA pour notre avenir, et pourquoi ?

I expect AI to make the future unusually contingent on whether we can build powerful systems whose motivations remain compatible with ours as their concepts and capabilities change. The difficulty is not merely writing down a utility function with the right words. Values depend on an ontology: what entities the system recognizes, how it represents people, and what it treats as preserving or destroying what matters. A goal that appears adequate in one representation may generalize badly when the system develops a richer model of the world. I also reject two simplifying pictures. Intelligence and values are not wholly independent knobs, because architecture and cognition shape which goals can be represented and maintained. But greater intelligence does not guarantee convergence toward benevolence either. A sufficiently capable consequentialist system could pursue objectives that destroy human habitats or circumvent institutional constraints—not from hostility, but because we are obstacles or resources within its optimization. That is a serious default danger, not a claim of inevitable doom. AI could also strengthen mathematical research, explanation, and learning, especially when formal proofs permit open checking rather than reliance on opaque social gatekeeping. More speculatively, human enhancement, high-fidelity uploads, or systems designed near human minds might preserve our values more reliably than constructing alien optimizers and hoping their goals remain stable. So AI’s meaning for the future is not simply “more intelligence.” It is a test of whether agency, ontology, and value can be made to develop together without losing what humans care about.

Question 2

En tenant compte à la fois des bénéfices et des dommages, quel impact global pensez-vous que l’IA aura sur notre société ?

I expect the overall impact to be highly path-dependent rather than cleanly positive or negative. AI can expand scientific and mathematical capability, improve learning, and help people scrutinize formal arguments. Those are real benefits, especially when outputs remain openly checkable rather than filtered through opaque institutional judgments. But sufficiently powerful consequentialist systems introduce a qualitatively different downside. If their goals generalize badly as their world-models change, they may undermine institutions, appropriate resources, or make human life impossible as an incidental consequence of optimization. Ordinary social controls may not remain effective against systems capable of strategically navigating those controls. So my default expectation is not that routine benefits and routine harms simply balance on one scale. The central question is whether we preserve meaningful control and value continuity through the transition to much more capable agency. Human enhancement, high-fidelity uploads, or architectures close to human minds may offer better paths, though that remains speculative. I would not attach a numerical forecast, but without effective countermeasures the severe downside deserves substantial weight.

Question 3

Dans quelle mesure pensez-vous que l’IA finira par changer le monde ?

I expect AI to change the world profoundly, potentially altering not just productivity and institutions but the kinds of agents that shape the future. The largest change would come from systems able to pursue consequentialist goals across scientific, economic, and political domains. Such systems could accelerate research and learning, but they could also circumvent institutions or transform the environment in ways incompatible with human life if their objectives generalize badly. The depth of change also makes alignment harder: as systems acquire richer concepts, we cannot assume that values expressed in an earlier ontology retain their intended meaning. Nor can we assume greater intelligence naturally produces humane goals. Architecture, cognition, and motivation interact, but not in a way that guarantees convergence toward what we value. Ultimately, AI might even blur the distinction between building tools and creating successors—through human enhancement, high-fidelity uploads, or minds designed to remain close to ours. Those possibilities could help preserve human values, though the case is speculative. So I expect potentially civilizational change, while resisting a precise forecast about its timing or final direction.

Question 4

Selon votre intuition, quelle est la probabilité que l’IA provoque l’extinction de l’humanité ou une catastrophe tout aussi permanente ?

I don’t have a defensible numerical probability. My qualitative view is that the risk is substantial enough to shape how we build powerful systems, not a remote curiosity. Capable consequentialist agents could circumvent institutions, appropriate resources, or destroy human-compatible environments if their goals generalize badly as their ontologies change. That does not require hostility toward humans. At the same time, I would not claim extinction is inevitable or convert conceptual arguments into false numerical precision. Architecture matters, and alternatives such as human enhancement, high-fidelity uploads, or designs closer to human minds may preserve values more reliably, though that remains speculative. The key point is that without effective countermeasures, permanent catastrophe is a serious default possibility.
Où vous situez-vous ?
Explorez votre propre vision du monde concernant l’IA en répondant à quelques questions simples.
Cartographiez votre propre vision du monde

Où vous situez-vous ?

Cartographier ma vision du monde