AlphaGo engineer leaves Google DeepMind, says LLMs don't reason

Graepel's argument rests on a distinction he draws between two modes of thought, borrowed from Daniel Kahneman: fast, gut-level System 1 and slow, deliberative System 2. AlphaGo, he writes, had both—its neural networks supplied hunches, and its search machinery tested those hunches against possible futures. That combination is what produced move 37, the play that looked like a glitch but helped AlphaGo beat Lee Sedol 4-1. Today's large language models, in his account, only have the hunch half.

He points to a deeper gap. AlphaGo kept a game tree—a structured record of the variations it had considered, annotated with judgments from its networks—and updated it as its reasoning progressed. Chatbots, by contrast, keep no such ledger of hypotheses, confidence, and open questions. Graepel argues this makes it hard to pinpoint what went wrong when a system errs, a problem he says matters most in medicine, engineering, and research.

His proposed alternative is a system that maintains an epistemic state and treats reasoning as a sequence of moves that change it—deducing, decomposing problems, and deciding what to do next. Whether such systems can handle the messiness of the real world, where the state of affairs is only partly known and consequences are uncertain, is the open question his argument leaves for others to answer.