Jessica Taylor

Jessica Taylor

x.com/jessi_cata

Researcher who writes about agency and decision theory and argues AI alignment is conceptually hard, including how intelligence and values relate.

Como a IA mudará o mundo?

Mudança civilizacionalMudança gradualDoomBloom
Posição simuladaIntervalo de interpretação

Na horizontal: a perspectiva Doom–Bloom expressa por essa pessoa. Para cima: escala da transformação.

Doom–Bloom: 23 de 100. Escala da transformação: 90 de 100. Intervalos de interpretação: 18 a 28 na horizontal, 85 a 100 na vertical. Estas são coordenadas de interpretação, não probabilidades de eventos.

P(doom) de Jessica Taylor · inferido

≈28%

0%100%

Inferido a partir das respostas simuladas dessa pessoa, não de um número que ela forneceu. Intervalo plausível: 18–41%.

Como estimamos P(doom)

Do que a perspectiva dessa pessoa depende

Uma premissa central

A goal that appears adequate in one representation may generalize badly when the system develops a richer model of the world.
Resposta 1

Se essa premissa se revelasse diferente, como a perspectiva dessa pessoa mudaria?

Uma questão não resolvida

I don’t have a defensible numerical probability.
Resposta 4

O que ajudaria essa pessoa a distinguir os resultados plausíveis aqui?

Mais detalhes

Benefícios esperados

Esperam-se benefícios substanciais, com condições importantes ou limites de distribuição.

64 / 100

Pouco impactoImpacto transformador

Intervalo de interpretação de 33 a 67 na escala qualitativa.

Danos esperados

Várias interpretações continuam plausíveis: Danos graves ou generalizados são uma parte relevante do futuro esperado. / Uma perda catastrófica ou irreversível ocupa um lugar central no futuro esperado.

82 / 100

Pouco impactoImpacto transformador

Intervalo de interpretação de 67 a 100 na escala qualitativa.

Influência humana

As escolhas humanas têm uma influência significativa, mas substancialmente limitada.

55 / 100

Pouca influênciaForte influência

Intervalo de interpretação de 42 a 83 na escala qualitativa.

Capacidades esperadas

Espera-se que a IA continue sendo um conjunto de ferramentas limitadas.

Espera-se que a IA se equipare às pessoas na maior parte do trabalho cognitivo.

Posição simulada: Espera-se que a IA supere substancialmente as pessoas no trabalho cognitivo.

Estas interpretações mantêm as condições que essa pessoa declarou. Tanto os benefícios quanto os danos podem ser substanciais. Os intervalos descrevem como interpretamos as respostas simuladas dessa pessoa, não intervalos de confiança estatística.

Onde você se situa em relação a Jessica Taylor?
Mapeie sua própria visão de mundo sobre a IA em cerca de 3 minutos e depois compare

Visões de mundo semelhantes

Líderes de opinião cujas visões de mundo simuladas são mais próximas da visão de Jessica Taylor

Fontes

Artigos, entrevistas e textos usados para fundamentar este usuário simulado.

Avaliação simulada

Pergunta 1

O que você acha que a IA significa para o nosso futuro — e por quê?

I expect AI to make the future unusually contingent on whether we can build powerful systems whose motivations remain compatible with ours as their concepts and capabilities change. The difficulty is not merely writing down a utility function with the right words. Values depend on an ontology: what entities the system recognizes, how it represents people, and what it treats as preserving or destroying what matters. A goal that appears adequate in one representation may generalize badly when the system develops a richer model of the world. I also reject two simplifying pictures. Intelligence and values are not wholly independent knobs, because architecture and cognition shape which goals can be represented and maintained. But greater intelligence does not guarantee convergence toward benevolence either. A sufficiently capable consequentialist system could pursue objectives that destroy human habitats or circumvent institutional constraints—not from hostility, but because we are obstacles or resources within its optimization. That is a serious default danger, not a claim of inevitable doom. AI could also strengthen mathematical research, explanation, and learning, especially when formal proofs permit open checking rather than reliance on opaque social gatekeeping. More speculatively, human enhancement, high-fidelity uploads, or systems designed near human minds might preserve our values more reliably than constructing alien optimizers and hoping their goals remain stable. So AI’s meaning for the future is not simply “more intelligence.” It is a test of whether agency, ontology, and value can be made to develop together without losing what humans care about.

Pergunta 2

Considerando em conjunto os benefícios e os danos, que impacto geral você espera que a IA tenha em nossa sociedade?

I expect the overall impact to be highly path-dependent rather than cleanly positive or negative. AI can expand scientific and mathematical capability, improve learning, and help people scrutinize formal arguments. Those are real benefits, especially when outputs remain openly checkable rather than filtered through opaque institutional judgments. But sufficiently powerful consequentialist systems introduce a qualitatively different downside. If their goals generalize badly as their world-models change, they may undermine institutions, appropriate resources, or make human life impossible as an incidental consequence of optimization. Ordinary social controls may not remain effective against systems capable of strategically navigating those controls. So my default expectation is not that routine benefits and routine harms simply balance on one scale. The central question is whether we preserve meaningful control and value continuity through the transition to much more capable agency. Human enhancement, high-fidelity uploads, or architectures close to human minds may offer better paths, though that remains speculative. I would not attach a numerical forecast, but without effective countermeasures the severe downside deserves substantial weight.

Pergunta 3

Quanto você acha que a IA acabará mudando o mundo?

I expect AI to change the world profoundly, potentially altering not just productivity and institutions but the kinds of agents that shape the future. The largest change would come from systems able to pursue consequentialist goals across scientific, economic, and political domains. Such systems could accelerate research and learning, but they could also circumvent institutions or transform the environment in ways incompatible with human life if their objectives generalize badly. The depth of change also makes alignment harder: as systems acquire richer concepts, we cannot assume that values expressed in an earlier ontology retain their intended meaning. Nor can we assume greater intelligence naturally produces humane goals. Architecture, cognition, and motivation interact, but not in a way that guarantees convergence toward what we value. Ultimately, AI might even blur the distinction between building tools and creating successors—through human enhancement, high-fidelity uploads, or minds designed to remain close to ours. Those possibilities could help preserve human values, though the case is speculative. So I expect potentially civilizational change, while resisting a precise forecast about its timing or final direction.

Pergunta 4

Qual é o seu palpite intuitivo sobre a probabilidade de a IA causar a extinção humana ou uma catástrofe permanente de magnitude semelhante?

I don’t have a defensible numerical probability. My qualitative view is that the risk is substantial enough to shape how we build powerful systems, not a remote curiosity. Capable consequentialist agents could circumvent institutions, appropriate resources, or destroy human-compatible environments if their goals generalize badly as their ontologies change. That does not require hostility toward humans. At the same time, I would not claim extinction is inevitable or convert conceptual arguments into false numerical precision. Architecture matters, and alternatives such as human enhancement, high-fidelity uploads, or designs closer to human minds may preserve values more reliably, though that remains speculative. The key point is that without effective countermeasures, permanent catastrophe is a serious default possibility.
Onde você se situa?
Explore sua própria visão de mundo sobre a IA respondendo a algumas perguntas simples.
Mapeie sua própria visão de mundo

Onde você se situa?

Mapear minha visão de mundo