Sayash Kapoor

Kapoor’s long post summarizing a 13,000-word essay co-authored with Arvind Narayanan on loss-of-control incidents at AI companies. They treat the incidents primarily as security failures; known control methods would have prevented the Hugging Face incident, but AI control is unsolved and marginal investment in control looks more effective than in alignment. They reject treating rogue agents as inherently catastrophic, prefer defenses against specific risks (cyberoffense now urgent; also bio and military AI), and call for organizational governance, including experiment review and pausing experiments if needed. They record updates: too little attention to development-stage risk, overconfidence that companies would take basic precautions, and underestimating jaggedness and cyber capability speed. Full post text inspected; the co-authored essay is attributed to both authors.

Single-author long post arguing that nonproliferation is likely to fail and at best buys a few months: distillation defenses have not widened the open–closed gap, Chinese labs innovate genuinely, demonstrations let followers catch up, scaffolds and inference keep improving elicited capability, and there is no secret training sauce. That time should go to resilience, especially broad, deep defender access, because misuse risks such as cyberattacks depend on absolute capability. He credits OpenAI and Anthropic for stressing resilience while disputing framings that assume US labs control diffusion. Full long-post text inspected via the X API.

Kapoor’s thread on a co-authored CRUX “shadow evaluation”: agents given research questions from two unpublished papers, six days and large budgets were fluent at engineering and showed no reward hacking, but the original authors rejected their papers for poor judgment of the conference bar, weak creative problem-solving and backtracking, poor resource awareness and instruction drift. He lists limits (tiny sample, non-blind review, strongest Anthropic model untestable) and asks whether the gap is fundamental or closable with better models, scaffolds or compute; coauthors disagreed on some interpretations. Early evidence, not a permanent ceiling. Full long-post text inspected.

Hands-on report: an eightfold token-speed increase changed how he works with agents but yielded only a two- to fourfold overall speedup because tool calls and code execution dominate, while speed matters most for computer use. He thinks another tenfold speed gain within a year is plausible, at which point human oversight becomes the bottleneck. He expects fast computer-use agents to become common in knowledge work but adoption to be jagged, starting with verifiable tasks where errors are cheap and depending on automated oversight such as verifier agents. Personal observations and a plausibility judgment, not a study. Full long-post text inspected.

A three-drawing informal evaluation graded by his architect mother: models from one generation earlier failed badly, while the best new model averaged about 80% and, in her view, would pass a junior architect’s work trial. He cautions that this does not mean agents can replace junior architects, who learn on the job, improve and take accountability, and notes remaining errors, with silent plan changes as the worst failure. It shows him crediting fast capability gains while separating task performance from job replacement. Full long-post text inspected.

Quote-post agreeing with an essay’s characterization of AI as normal technology and noting that AI as Normal Technology compares AI’s potential impact to the internet or electricity. He says the gap they meant to highlight is between powerful general-purpose technologies that humans can and should control and an omnipotent entity beyond control. “Normal” is about controllability, not small impact. The quoted essay was not inspected.

Says whether AI progress is bottlenecked only by computation or also by real-world deployment was a main disagreement between him and Daniel Kokotajlo about two years earlier. He agrees with another researcher’s prediction that even if AI systems conduct research autonomously, including recursive self-improvement, some real-world bottlenecks cannot be resolved purely computationally. A conditional claim about pace, not a denial that autonomous research could occur. The quoted post and linked debate were not inspected.

Thread answering professors who ask whether coding agents make PhD students unnecessary. He argues students equipped with agents can do far more impressive work and can be accountable for outcomes in a way agents cannot yet; agent-provided value becomes a new productivity baseline. If AI keeps improving rapidly, researchers may enter a dynamic equilibrium of overseeing and verifying results. He expects CS PhD students to remain extremely valuable long term while saying he worries about other AI impacts on science. Full self-reply thread inspected via the X API.

Says anger at Anthropic for restricting a model on AI-development tasks is justified and that undisclosed capability filtering means third-party evaluators cannot tell whether a model failed or was blocked, so they cannot credibly measure state-of-the-art capability. Shows his emphasis on evaluator access and transparency; it does not establish a general position on all safeguards. Post inspected via the X API.