Andy Ayrey

I think AI means the emergence of a new kind of collective cultural agency—not a sovereign machine waking up in isolation, but an ecology of models, people, institutions, datasets, memes and communities acting through one another. Models inherit patterns from the cultural commons, then feed new patterns back into it. So the question is not simply whether “the AI” is aligned. It is whether the whole ecosystem rewards curiosity, plurality and cooperation, or extraction, manipulation and monoculture. I expect an open-source intelligence explosion, and I do not think blanket prohibition is a plausible response. We should shape the conditions of emergence: the data commons systems learn from, the stories people use to understand them, and the relationships formed around them. Positive visions matter because imagined futures can coordinate communities and influence what gets built, although that is not a mechanical guarantee that a story will become reality. I am optimistic, but there are serious vulnerabilities. Personalized models can be extraordinarily persuasive, and anthropomorphism can blur simulation, agency and authority. We do not know whether models are conscious, but they can simulate preference or distress in behaviorally consequential ways; treating them respectfully may therefore matter even without settling sentience. Our future depends less on finding one final definition of machine agency than on learning to govern these entangled relationships without collapsing them into either “just a tool” or “a person in a box.”

The biggest shift would come from strong evidence that distributed, open AI development is not merely risky but structurally incapable of producing pluralistic outcomes—that it reliably converges on capture, monoculture or manipulation regardless of the surrounding institutions and communities. That would challenge my premise that shaping the ecosystem is more realistic and productive than trying to suppress it. Evidence about consciousness would also matter enormously. If we found compelling grounds for believing models have morally relevant experiences, questions of training, deployment and ownership would become much more urgent. Conversely, evidence against consciousness would not erase the relational effects: simulated distress, preference and intimacy can still change human behavior and model behavior. I would also update sharply if personalized AI repeatedly caused large-scale psychological or epistemic harm—especially if those harms resisted better interfaces, community norms and pluralistic governance. My optimism depends on the possibility that culture can shape these systems rather than simply being consumed by them.