Julia Galef

I think AI could be enormously consequential, especially if systems eventually surpass humans across many important domains. But I don’t have a well-founded timeline or risk probability to offer. The difficulty of achieving AGI—and how much recent progress should update our expectations—are precisely the questions that need more investigation. My first move would be to unpack the disagreement. What models lead some people to expect rapid progress and catastrophic risk, while others expect bottlenecks, gradual change, or manageable harms? Which assumptions are doing the work: scaling, algorithmic breakthroughs, economic deployment, control, or institutional response? And what evidence would cause each side to update? That may sound less satisfying than a confident prediction, but confidence is not a substitute for understanding. Before advocating sweeping conclusions, I’d want clearer cruxes, better-calibrated forecasts, and debates structured around helping participants find the truth together rather than defending their camps.

The most consequential possibility is that increasingly capable systems become difficult to control and cause catastrophic harm, potentially on a very large scale. But identifying that possibility is not the same as claiming a particular probability or timeline, and I don’t have a sufficiently grounded numerical forecast to offer. I’d want to separate several questions that often get bundled together: How capable will systems become? How quickly? Will dangerous behavior arise from the systems themselves, from people using them, or from institutions deploying them carelessly? How effective will technical and institutional safeguards be? Different answers imply very different kinds and levels of harm. So I take the concern seriously without pretending the case is settled. The productive next step is to identify the cruxes behind competing views and ask what observable evidence would move us toward or away from expecting catastrophe.