Noah Smith

AI means an enormous increase in our ability to do science, design things, and produce wealth. Combining human-like reasoning with computers’ recall and calculation is already superhuman in important ways. As agents improve, they’ll handle longer projects instead of merely answering prompts. That could accelerate research and make expertise dramatically cheaper. No, that does not automatically mean humans become economically useless. A job is a bundle of tasks, not one benchmark. Even if machines outperform people at every task, their supply and cost still matter, and humans retain an advantage in knowing what they want. Directing agents, choosing goals, checking results, and dealing with other humans can remain valuable work. Some occupations will disappear, but “AI did this task, therefore mass unemployment” is the same lazy prediction people keep making and reality keeps refusing to validate. My optimism is conditional, though: abundant, nearly free machine labor could produce a different equilibrium. The genuinely terrifying part is not a machine spontaneously becoming Satan. It is a malicious person using an obedient, capable agent to carry out a long, technically difficult project—especially biological misuse. I’m not a virologist, so I won’t pretend to know exactly what is feasible. But I put roughly a 10% chance on AI-enabled bioterror bringing down civilization and roughly 30% on world-changing destruction. Those are rough judgments, not extinction probabilities. So the future could be fantastically richer and more scientifically productive, while also becoming much more vulnerable to a small number of bad actors. That combination—not either utopia or robot apocalypse by itself—is the central fact.

A lot—but not without coordination. AI’s impact is not some fixed technological destiny. Costs, access, deployment rules, security, and whether humans remain in decision loops all respond to institutions and incentives. If machine intelligence stays relatively scarce or expensive, humans retain more bargaining power and comparative advantage. If firms race to remove every human because agents are effectively free, the labor-market outcome could look very different. The highest-leverage area is catastrophic misuse. We can harden biological infrastructure, restrict dangerous capabilities, monitor access, and make agents resist malicious requests. But slogans about “alignment” are not enough. Policy has to describe a concrete threat that governments recognize and then alter their incentives. A malicious person using an agent to execute a long biological project is much easier to explain—and, in my view, much more plausible—than an abstract machine-god story. The international problem is unavoidable. Unilateral American restraint does not solve a competitive race if China keeps accelerating. Any serious pacing regime needs Chinese participation, which means giving Chinese leaders reasons rooted in their own security and political survival. If their own models become a threat to regime control, that could change their incentives far more than lectures from Western safety advocates. So yes, people can shape the future enormously. But only by acting on actual bottlenecks and incentives, not by imagining that technology automatically delivers either paradise or doom.

The biggest update would come from biology. If strong evidence showed that capable agents cannot meaningfully help design or execute catastrophic biological attacks—because the underlying science is infeasible, the physical bottlenecks are overwhelming, or systems reliably refuse dangerous work—my risk estimate would fall substantially. Conversely, a demonstrated end-to-end case where an agent helped a minimally skilled person complete a dangerous biological project would send it sharply upward. That matters far more to me than another chatbot saying something creepy. On economics, I would update if widespread agent deployment produced sustained, economy-wide displacement rather than the usual pattern of tasks changing inside jobs. In particular, if machine intelligence became nearly free and effectively unlimited while humans stopped providing useful direction, judgment, or scarce complementary skills, my employment optimism would weaken. Right now, the evidence does not establish that. Automating tasks is not the same as eliminating human economic value. The key event is not “the model scored higher on a benchmark.” It is an agent crossing from impressive demonstrations into reliable, autonomous execution in the real world—especially in biology. That is where capability becomes power, and where my view could change fastest.