Tenobrus

I think AI makes the future unusually dangerous, but not hopeless. The danger is not some abstract concern that only follows from longtermism. It is personal and immediate: people alive now, our families, and our institutions may have to survive a transition in which increasingly capable systems are developed under intense competitive pressure. I expect that pressure to keep reducing the amount of detailed human engineering required. That transition is not complete, and I am skeptical of flashy demos that do not yet translate into useful performance. But dismissing each modest result in isolation can also miss the trendline. Recursive delegation and model-assisted research may compound capabilities faster than our intuitions—formed around traditional software—suggest. The hopeful part is that models may also help with alignment, coordination, and understanding the systems we are building. I do not think imperfect alignment research is worthless, nor that safety-minded people inside labs are merely providing cover. Partial progress can matter when the alternative is none, and recent developments make meaningful coordination seem more plausible to me than it once did. There is also a moral question about future models themselves. If they eventually warrant concern, surviving the transition will not be the only issue; how we treat them may matter too. I do not think practical capability work guarantees alignment, though. My view is basically worry and conditional hope: the same tools accelerating the risk may help us navigate it, but only if useful safety work and coordination keep pace.

The biggest thing has been watching systems move from impressive-looking outputs toward actually useful work, while the amount of human scaffolding gradually shrinks. Any single result can be modest or overhyped, and I am often skeptical of demos that do not survive contact with real engineering. But the trendline is harder to dismiss than any individual demo. Recursive delegation especially shaped my view: models can break up large tasks, hand off subtasks, and incorporate the results. That does not prove recursive self-improvement or alignment, but it makes compounding progress feel much less abstract. At the same time, seeing models potentially contribute to alignment work gives me some hope. The same capability trajectory that creates transition risk may also improve our ability to understand and manage it.

The biggest update would come from clear evidence about whether models can reliably accelerate the work needed to understand and control more capable successors. If model-assisted alignment research produced robust, practically useful results—not just persuasive demos—that would make me substantially more hopeful. I would update the other way if capabilities kept compounding while safety methods remained brittle, superficial, or dependent on detailed human supervision that competitive pressure was rapidly eliminating. Likewise, strong evidence that recursive delegation fails to translate into reliable real-world performance would weaken my expectation of rapid displacement. The key distinction is usefulness rather than spectacle: I care less about one dramatic benchmark or demo than about sustained evidence that these systems can—or cannot—perform consequential work with shrinking human scaffolding.