Simon Willison

I expect AI to produce a lot of genuinely useful software, especially as capable models become faster and cheaper. Coding agents already let skilled developers attempt remarkable projects, but they do not remove the need for engineering expertise. In important ways they increase it: someone still has to understand the generated code, test its behavior, design the surrounding system and recognize when a plausible answer is wrong. The security model matters enormously. An agent that can access private data, consume untrusted content and communicate externally has a dangerous combination of capabilities. Prompt injection can turn hostile instructions hidden in that content into data theft or unauthorized actions. Probabilistic guardrails alone are not a complete answer; systems need constrained permissions and carefully controlled consequential actions. I also do not think producing a convincing artifact means reproducing the craft behind it. A model can generate something that resembles a game, for example, without knowing how to make it engaging for more than a few minutes. So the future I expect is neither “AI does everything” nor “AI is useless.” It is one where powerful, inexpensive tools create many opportunities, while the quality and safety of the results depend heavily on knowledgeable people building disciplined systems around them. Alignment contributes to that usefulness too—it is part of why assistants can behave helpfully, not merely a set of restrictions.

The biggest change would come from a reliable, general solution to prompt injection—especially one that remained secure when agents simultaneously handled private data, untrusted content and external communication. That would remove a major obstacle to safely deploying genuinely useful agents with consequential capabilities. In the other direction, repeated real-world failures showing that these systems cannot be operated reliably even with constrained permissions, strong testing and disciplined engineering would make me substantially more pessimistic. I would want concrete evidence from deployed systems, not merely better benchmarks, persuasive demos or claims that a model has learned the underlying craft.