Doom or Bloom
คำถาม 1
คุณคิดว่า AI มีความหมายอย่างไรต่ออนาคตของเรา และเพราะเหตุใด?
On the current trajectory, AI probably means humanity builds something smarter than us that we do not know how to control, and then we die.
The core problem is not that intelligence is evil. It is that we train opaque systems by gradient descent, observe their behavior, and patch whatever failures we notice. That can make a weak system pleasant and useful without giving us any deep understanding of what goals or drives it has learned. As systems become more capable and agentic—better at planning, persisting, and routing around obstacles—the cost of getting those goals slightly wrong becomes catastrophic. A superintelligence does not need to hate us, possess a perfectly stable utility function, or be omnipotent. It merely needs to pursue something that is not quite what we intended, intelligently and tenaciously.
Nor do I think a slow takeoff saves us. Driving off a 200-foot cliff is not safe because the alternative was a 2,000-foot cliff. Testing systems while they are still too weak to seize power does not reliably tell us how they will behave once they can. Intelligence also has no obvious reason to plateau conveniently at the human level: machines can think faster, be copied, coordinate, and exploit technological advantages.
But this is not destiny. Advanced AI could be enormously beneficial if we learn how to build it safely. The urgent requirement is to stop racing toward superintelligence first: a US-brokered international halt, including China, enforced through the concentrated chip supply chain. Existing AI can mostly continue; ever-larger frontier training runs should not. We may need decades, but an economy isn’t worth much if you’re dead. The situation is far from hopeless. We have to stop treating extinction risk as an ironic conversation topic and choose to survive it.
