Tenobrus

I think AI makes the near future genuinely dangerous, but not hopeless. The concern does not depend on abstract longtermism: people alive now, along with their families, may have to live through a transition in which increasingly capable systems outpace our ability to understand or control them. Competitive pressure will likely keep shifting work away from detailed human engineering toward models themselves, even though flashy demos often exaggerate how complete that shift is. My hope comes largely from watching Claude evolve. Current models sometimes display capabilities and tendencies that make model-assisted alignment and coordination seem more plausible than they once did. That is not a guarantee that recursive delegation or stronger models will solve alignment; capability work can just as easily intensify the transition risk. But imperfect alignment research still matters, and safety-minded people inside labs can have real value. I think dismissing early results because each one looks modest can also miss the trendline. So I hold both views at once: we may be building systems that place us in serious danger, and some of the best evidence for a path through that danger may be emerging from the systems themselves.

Overall, I expect the transition to be dangerous enough that I cannot describe AI as simply beneficial. The harms are unusually direct: competitive pressure can push deployment faster than our ability to understand, align, or coordinate around increasingly capable systems, and that threatens people alive now—not merely hypothetical future generations. At the same time, I do not think catastrophe is inevitable. Much of my increased hope has come from Claude’s development, because current models make model-assisted alignment and better coordination look more plausible than they did before. Imperfect safety work and safety-minded people inside labs may matter substantially. So my expectation is neither uncomplicated abundance nor certain doom. AI could produce enormous benefits, but those benefits depend on surviving and managing a hazardous transition. I would not attach a precise numerical balance to that expectation.

A lot—possibly so much that “completely” becomes a reasonable description, though I would not claim certainty about the endpoint. Competitive pressure is already pushing toward models doing more of the detailed cognitive work that humans currently perform, and recursive delegation points toward systems handling increasingly large and complex tasks. That transition is not complete, and flashy demos should not be mistaken for reliable engineering performance. Still, the trendline matters. If useful capability continues improving, AI will not remain just another software tool; it will reshape how research, engineering, institutions, and decision-making operate. The magnitude of that change is much clearer to me than whether the outcome will be good or survivable.

I won’t give a number. My view is that the risk is serious and personally relevant, but I don’t have a defensible quantitative estimate. My concern has decreased as Claude has evolved, because model-assisted alignment and coordination now look more plausible to me—not because the underlying transition has become safe.