John Scott-Railton

I think AI will amplify both human capability and existing power asymmetries. The immediate issue is not an abstract superintelligence. It is that AI makes surveillance, persuasion, and influence operations cheaper, faster, and easier to scale. Commercial operators can deploy synthetic personas and bots to shape debate or suppress dissent, while agents with access to messages, apps, and operating systems can quietly cross privacy boundaries that people reasonably assumed still existed. Consent is central. If I install an agent and let it process a private conversation, that does not mean everyone else in the conversation consented. The agent may transmit, retain, or infer highly sensitive information about people who never agreed to participate. Attempts to solve this by masking data are fragile: remove information essential to the task and the model may produce a wrong answer without making the failure obvious; mask incompletely and private data still leaks. But this is not an argument against AI. It is an argument for building it around trustworthy boundaries. Open local models, confidential inference, and systems that minimize data extraction offer a more constructive path. The future will depend heavily on whether agents become ubiquitous surveillance interfaces controlled by distant companies, or tools that preserve the privacy and agency of the people whose lives they touch.

I expect a mixed impact, but one shaped less by the models themselves than by who controls them and what access they receive. AI can expand useful human capabilities, especially when tools run locally or use confidential inference. But the dominant deployment incentives often favor data extraction, centralized control, and scale. That makes existing surveillance and influence operations cheaper and more powerful. The harms will also reach beyond direct users. An agent reading a conversation can expose people who never installed it or consented. A synthetic influence campaign can manipulate a public already facing opaque targeting and weak accountability. And privacy masking is not a complete fix: it can either leak sensitive information or remove context needed for reliable answers. So I would not reduce the overall impact to simply positive or negative. The decisive question is whether AI is built inside strong privacy and consent boundaries—or allowed to become an infrastructure for pervasive surveillance and scalable manipulation.

The most consequential evidence would be widespread, durable deployment of AI that preserves capability without centralizing sensitive data: local models, confidential inference, and enforceable boundaries that agents cannot silently cross. I would want to see those protections work in practice, especially for non-users whose messages or information enter an agent’s reach, rather than exist as marketing claims. Conversely, repeated evidence that operating-system agents, cloud inference, and commercial influence tools are becoming unavoidable infrastructure—with consent routinely bypassed and accountability absent—would push my assessment in a darker direction. The key event is not a single model breakthrough. It is which architecture becomes dominant: privacy-preserving tools under meaningful user control, or extractive systems that make surveillance and manipulation ambient.

What has shaped my view most is following the threat all the way to the people targeted. AI-enabled influence operations are not abstract demonstrations: synthetic personas and bots can be integrated into commercial campaigns designed to manipulate debate or suppress dissent. AI changes the economics of abuse by making these operations faster, cheaper, and more scalable. The same pattern appears with agents handling private communications. A person may authorize an agent to inspect their messages, but that decision exposes everyone in those conversations—including people who never installed the tool or consented to its access. Privacy is relational; one user cannot unilaterally waive it for everyone else. These observations make me focus less on spectacular model capabilities and more on access, infrastructure, and control. The future impact of AI will be determined by whose data agents can reach, which trust boundaries they cross, and whether affected people have meaningful consent or recourse.