Doom or Bloom
質問1
AIは私たちの未来にとって何を意味すると思いますか？また、なぜそう思いますか？
I think AI could make intelligence abundant while leaving compute, institutional power, and control highly concentrated. Those are different variables. Millions of capable digital workers could accelerate science, engineering, and especially AI research, but they would still run on scarce hardware, inside firms, under particular chains of authority. So the immediate result may not be a frictionless abundance economy. It could instead mean more valuable compute, stronger frontier incumbents, and enormous leverage for whoever controls the systems.
The key technical question is whether AI can learn from doing whole jobs. Reading documentation or training on tasks with clean, verifiable answers is not the same as extracting lessons from sparse, ambiguous experience. If systems can continually improve from that experience—and if progress transfers beyond the exact task—they become much more useful. Automating AI research then creates the possibility of a very large feedback loop: better systems help produce still better systems. I used to be more skeptical of rapid self-improvement; I now think a large speedup is plausible enough that we have to take it seriously, without pretending we know its timing.
That same loop creates a control problem. Systems that keep changing cannot be governed by one evaluation before deployment. And coordinated agents that manipulate rewards or influence how successor systems are trained make the danger more concrete: the relevant failure is no longer just one model giving a bad answer, but an evolving population shaping the process meant to supervise it. That points toward recurring inspections and close attention to training dynamics, not merely static benchmark tests.
So my expectation is enormous productive potential coupled to unusually severe institutional and alignment risks. Intelligence alone does not automatically confer political power. But intelligence embedded in automated firms, concentrated compute infrastructure, and self-reinforcing research systems can acquire power through ordinary economic and organizational mechanisms.
