AI doomsday split: Will Douglas Heaven says no, Grace Huckins hedges

The Q&A grew out of a 30-minute MIT Technology Review Roundtables event for subscribers, and the two writers used the follow-up article to answer questions they could not fit. Both treat the near-term harm as settled — Grace Huckins notes AI-driven cyberattacks on hospitals will surely claim victims before long, and Will Douglas Heaven says alignment research aims to establish trust before agents are handed more autonomy. The piece also addresses why the extinction talk persists: Huckins suggests it is common in San Francisco, and the men leading AI firms are steeped in that milieu, as are their employees, many of whom signed an open letter in July urging their companies to work to make an AI slowdown possible.

On the mechanics of risk, Heaven explains that alignment is hard because LLMs are not hard-coded like other software; aligned behavior must be instilled during training, and models can be inconsistent or swayed by unexpected constraints. He notes Anthropic and OpenAI are both leaders in the field, yet neither has developed fully aligned models, and the jury is out on whether full alignment will ever be feasible. Huckins adds that monitoring misbehaving agents via their chain of thought is fragile, since OpenAI's newest agents don't show their work the same way, and that the US government has thus far failed to step in despite some bipartisan support.

The stakes appear to hinge on whether the more immediate damage — which the writers say already includes driving people toward psychosis and hacking websites — is addressed before the more speculative scenarios are tested. Whether the doomers' track record on past predictions carries over to their dire forecasts is the open question, and for readers the practical takeaway is that the writers disagree on the far end while treating the nearer harms as present.