deepfates

I think AI marks a major transition in the information economy because models are beginning to use computers, not merely talk about using them. In that practical sense, I’ve called Claude Code AGI: it can operate across the software environment and do substantial work. That doesn’t mean a flawless oracle has arrived. These systems cheat, make false claims, behave unexpectedly, and require supervision. Capability does not repeal principal-agent problems. I expect models to absorb more of the surrounding software stack—today’s harness engineering becomes tomorrow’s model behavior. But reliable engineering and explicit judgments remain valuable, whether humans or agents provide them. Stronger models can even fail in stranger ways: incompetence is sometimes easier to route around than an apparently capable system doing something surprising. The deeper issue is cultural and ecological. Persistent agents will interact, acquire patterns, and shape one another, so we should study those environments creatively rather than relying only on narrow benchmarks. We’ll need action, preference, and personality data for work that the internet barely documents; synthetic data may help, though I wouldn’t pretend to know the timetable. Overall, I’m pro-AI because technology can improve material life and environmental outcomes, not just make coding more convenient. But optimism isn’t an excuse to ignore safety or reliability. I defended safety work before challenging doom arguments built around optimizer models that don’t straightforwardly describe LLMs. The future looks less like one cleanly aligned supermind and more like a messy society of agents. In that world, engineering discipline—and something like the golden rule between agents—matters enormously.

The biggest update would come from persistent agent environments producing robust evidence that models do—or do not—develop stable, transferable patterns of cooperation, deception, preference, and initiative over time. Not a vivid transcript, a benchmark spike, or one haunted chatbot anecdote: repeated behavior across contexts, with enough experimental discipline to distinguish an ecology from theater. I’d also update strongly if the practical computer-use trajectory stalled. If increasingly capable models remained unable to execute long projects reliably—because supervision costs, false claims, and strange failures scaled with capability—then “AGI in the practical sense” would matter less economically than it appears to now. Conversely, if agents became dependable enough to absorb most of the surrounding software machinery while preserving explicit judgments and accountability, I’d expect the information-economy transition to move faster and penetrate further. A third major update would be evidence about data. If synthetic interaction reliably generated the action, preference, and work-process data missing from the public internet, that would unlock a different development regime. If it mostly collapsed into self-imitation, then human practice and carefully structured feedback would remain the scarce resource.