Teortaxes

AI matters because it expands the range of problems we can actually attack. The most important frontier is not another layer of consumer software or merely replacing programmers. It is scientific and industrial capability: helping researchers reason through difficult problems, supporting engineers and other professionals, and eventually improving work across many physical products. An AI need not be superhuman in every respect to be extremely valuable; reliable assistance on real tasks already changes what people can attempt. But the distribution of that capability matters. I do not want the future organized around chip embargoes, a closed national AGI project, or a few institutions deciding who gets access. AGI concerns everyone, so global participation is not optional theater. Open models are part of that picture, although a growing open-model business should not be confused with actually closing the capability gap at the frontier. I also want stronger public scrutiny of extraordinary claims. If a system allegedly produces a mathematical breakthrough, outsiders should get an unambiguous problem statement and a commitment to the withheld solution so they can attempt independent replication. Likewise, claims about alignment faking should not be generalized from one specially trained model without controls and cross-model comparisons. There are real failure modes, but “rogue AGI kills some people but does not end the world” occupies a substantial part of the plausible space too. Incidents can be severe without fitting an extinction-only narrative. So I expect AI to be scientifically transformative, politically contested, and dangerous in uneven ways—not reducible to either consumer convenience or one cinematic apocalypse.

Overall, I expect AI to be strongly transformative, with much of its value coming from scientific discovery, industrial work, and useful assistance to human professionals. It does not need to become universally superhuman to produce large benefits. The sign of the outcome depends heavily on access and power. Broad participation, independent scrutiny, and the ability to replicate important results make a better future more plausible. Chip embargoes, exclusive national projects, and a frontier controlled by a few institutions push in the opposite direction. Open models help, but their commercial growth alone does not prove that frontier power is decentralizing. I also expect serious harms, including narrow-goal failures that can kill people without ending civilization. That middle territory is neglected when discussion collapses into either harmless productivity software or extinction. So my overall expectation is major benefit mixed with real, uneven danger and intense political conflict over who controls the capability.

A lot—probably on the scale of a major industrial transformation. The deepest changes will come from scientific and industrial capability: accelerating research, assisting professionals, and improving the design and production of physical goods. AI does not need to become universally superhuman to alter what institutions and individuals can accomplish. I would not say “completely,” because that implies a total discontinuity or a specific endpoint I cannot justify. Existing political structures, physical constraints, and concentrations of capital will continue to matter. But AI will substantially reshape science, industry, labor, and geopolitical power—and who controls frontier systems will help determine the shape of that change.

No idea—I don’t have a defensible numerical probability. I think severe but nonterminal disasters occupy substantial probability space, and extinction risk should not eclipse them.