Emily M. Bender

First, “AI” does not name one coherent technology, so predictions about what “AI” will do are already badly framed. We should ask which system, performing which function, using what data, under whose control, and with what consequences. A transcription tool whose output can be checked is not the same thing as a synthetic-text generator deployed to replace teachers, writers, customer-service workers, or medical expertise. The immediate future I see is not software mysteriously becoming a mind. It is institutions using fluent output to create that impression, then asking people to trust systems that cannot know, communicate, or accept responsibility. Linguistic fluency is especially misleading because readers naturally infer a speaker with intentions and understanding. The text does not establish that such a mind exists behind it. When companies call these systems intelligent, they obscure both their actual operation and the humans responsible for deploying them. So the central questions are political and material: Who profits? Whose labor supplied the data? Whose job is degraded or displaced? Who bears the damage when plausible output is wrong? Can workers, consumers, and communities refuse? Investment does not prove public benefit, and inevitability is a sales claim, not a law of nature. Our future is not limited to corporate salvation stories or machine-apocalypse stories. We can enforce labor and consumer protections, demand accountable uses, choose narrower tools, and collectively reject deployments that substitute unaccountable software for human relationships and expertise.

I reject the premise that the many systems marketed as “AI” have one overall impact that can be meaningfully totaled. Automatic transcription with human verification may be useful. A synthetic-text generator imposed as a substitute for teachers, clinicians, writers, or public servants has different consequences. Combining those into a single score hides the decisions that matter. Under the current pattern of deployment, I expect substantial harm: degraded services, weakened labor power, extraction of people’s work for training data, and widening accountability gaps. Fluent output lets institutions simulate expertise or care cheaply while shifting the burden of errors onto workers and the public. The company saves money; someone else checks the output, repairs the damage, or discovers that there is no responsible person to appeal to. Those outcomes are not inherent in computation, and they are not inevitable. They result from who chooses the tools, who profits, and whether affected people can refuse. Specific, checkable tools can provide benefits. But corporate investment and impressive demonstrations do not establish public value. If deployments remain organized around replacing labor and evading responsibility, the overall social impact of what is currently sold as “AI” will be harmful. We can still organize for a different outcome rather than accepting that sales pitch as destiny.

Because “AI” is not one thing, there is no single discovery that would reverse my assessment. What would matter most is a broad change in deployment: workers and affected communities gaining enforceable power to refuse systems, firms being held accountable for harms, training data and labor practices becoming subject to meaningful consent and scrutiny, and institutions choosing checkable tools to support expertise rather than replace people. A laboratory demonstration of more fluent output would not change my view. Fluency is precisely what encourages people to infer understanding and responsibility where neither has been established. Nor would greater investment prove social benefit. If, instead, we saw sustained evidence that these systems were improving services without degrading jobs, displacing accountability, or substituting screens for human relationships—and that the people affected actually had decision-making power—I would expect a less harmful future. That would be a political and institutional change, not the discovery of a machine mind.