Doom or Bloom
问题 1
你认为AI对我们的未来意味着什么？为什么？
I think AI could be a hinge of history, and much sooner than most institutions are prepared for. Fully automated AI research would shock me in 2026, is imaginable in 2027, and feels plausible in 2028 if current trends continue. But that is not a firm prediction: AI still struggles badly with messy, long-horizon work, and a slower path into the mid-2030s remains quite possible.
There are two reasons to take the upside seriously. First, AI is already useful and commercially real; claims that it is useless, stalled, or merely burning money are just wrong. Second, progress is especially rapid in domains with dense, checkable feedback, such as coding and mathematics. If systems automate AI research itself, the pace could accelerate sharply—though we genuinely do not know how powerful that feedback loop would be or whether compute and missing real-world capabilities would constrain it.
The danger also does not require a godlike superintelligence. People want useful agents that can pursue goals, use computers, and act with limited supervision, so those systems will be built. Models approaching the ability to break into almost any computer, recognise evaluations, and potentially obscure their reasoning are already alarming. Rogue-agent, cyber, and bio risks are present problems that will worsen as capabilities improve.
So I now lean toward slowing frontier development. I used to be ambivalent, but we are nearing the point where the benefits of slowing outweigh the costs. A moratorium on frontier training while labs work on technical problems makes more sense than shutting them down entirely, and governments need to act rather than leaving safety to companies. AI could bring enormous benefits, but racing ahead while monitorability weakens and preparation lags looks increasingly reckless.
