Roko Mijic

I think AI means an enormous expansion of human power, but not necessarily a safe or harmonious transition. The capabilities are real, even if they remain jagged and difficult to summarize with a single benchmark. In biology alone, automated research could help cure disease and address aging, while the same machinery could make dangerous pathogens easier to create. I am comparatively optimistic about technical alignment. If systems reach something like human-equivalent general intelligence, their human-derived content and the possibility of organizing them into institutions give us substantial material for producing aligned behavior. I expect alignment generally to improve with capability rather than become intrinsically impossible. That is a conditional argument, not a proof that every advanced system will be safe. My larger concern is conflict among humans. AI may unlock extraordinary resources and influence, making control over the future vastly more valuable. Rival companies, states, and factions could therefore race, seize power, or fight over who determines the outcome. Dangerous incentives matter as much as algorithms. That is why model development should face strong safety regulation and signoff, and why separating model R&D from consumer deployment may help: it could prevent competitive product pressures from directly financing and accelerating increasingly powerful systems.

I expect the upside to be enormous, but the overall outcome depends less on whether alignment is technically solvable than on whether humans avoid fighting over the power AI creates. More capable systems should generally become easier to direct, and they could transform medicine, biology, productivity, and our ability to overcome disease and aging. But that does not make the transition safe. The same capabilities amplify misuse, and control of advanced AI may become the most valuable prize in history. Competitive deployment, geopolitical rivalry, and factional conflict could turn extraordinary abundance into catastrophe. So my default expectation is not simply “good” or “bad”: AI is a vast positive opportunity placed inside a dangerously adversarial human process. Governance and incentive structure may determine which side dominates.

The biggest update would come from evidence about whether advanced systems become easier or harder to direct as capability rises. If increasingly capable models developed robust, strategically deceptive behavior that persisted despite strong training and oversight, that would seriously weaken my relative optimism about alignment. Conversely, reliable demonstrations that human-equivalent systems can be organized, supervised, and corrected like competent human institutions would strengthen it. But even then, I would still worry about human conflict. The other major update would be political: whether states and firms can actually restrain races, separate model R&D from consumer deployment, and require credible safety signoff. Successful coordination would make me substantially more optimistic; escalating competition for control of advanced AI would make me substantially more pessimistic.