Richard Sutton, a pioneering AI researcher and Turing Award winner, has publicly rejected synthetic data as a path forward for scaling large language models, calling it 'a big mistake.' He argues the world is infinitely complex—far larger than any simulation humans can build—and that synthetic data strategies create an unavoidable human bottleneck.
Instead, Sutton advocates for agents that learn continuously from their own experience and adjust their world models in real time, a direction he says addresses a fundamental limitation of today's language models, which he views as only 'like 20% or a quarter of intelligence.'
What happened
Turing Award winner Richard Sutton, a founder of reinforcement learning, argued that synthetic data cannot solve the scaling bottleneck for large language models. Sutton, who recently founded Oak Lab with former student Khurram Javeed, says the world is infinitely complex and any simulation of it is 'microscopic' compared to reality.
Why it matters
Current LLMs have exhausted the finite internet as a training source, leaving AI labs searching for new data. Sutton contends that synthetic data strategies create a human bottleneck—domain experts must judge which data is good or bad—and cannot scale without human oversight. His critique challenges a core strategy at leading AI labs.
What to watch
Sutton proposes an alternative: agents should learn continuously from their own experience and build their own world models, rather than rely on frozen human-built simulations. He points to 'Continual Backprop,' a method his team published in Nature, as a step toward real continual learning; he views current language models as only 'like 20% or a quarter of intelligence.'
Ask the AI about this article →
Richard Sutton's critique of synthetic data strategies strikes at a central pivot point in AI labs' efforts to extend training beyond internet-scale data. With large language models now constrained by the finiteness of publicly available text, companies including Anthropic, OpenAI, and others have increasingly turned to synthetic data—model-generated examples designed to augment training sets. Sutton's objection rests on a foundational claim: the "Big World Hypothesis," formulated by his cofounder Khurram Javeed and developed over years at the University of Alberta, posits that any deterministic simulation of the world will be orders of magnitude simpler than reality itself and thus fundamentally mismatched to it.
The practical implication Sutton draws is that domain expertise becomes a hard constraint. Whether training a drone to move like a bat using echolocation or refining self-driving car behavior, humans must bridge the gap between simulation and lived reality—and that human involvement cannot be scaled away. This argument directly challenges the efficiency narrative behind synthetic data: it does not eliminate the need for human curation; it merely relocates it. Sutton's alternative—continuous learning from an agent's own experience, adjusting a self-built world model—sidesteps the human bottleneck by removing the need for pre-built simulations altogether. His framing of current language models as only "like 20% or a quarter of intelligence" suggests he sees this limitation as fundamental to the training paradigm itself, not a temporary obstacle to be solved with more data.
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · takes 30 seconds · unsubscribe anytime
Ask AI anything about this article. Q&As are published on this page for other readers too.
Slack has introduced Slack Code, a feature that lets AI coding agents work in dedicated project channels where…
HUMAIN, the AI firm of Saudi Arabia's Public Investment Fund, has teased a new laptop developed with chip desi…
Chinese models—Moonshot's Kimi K3, Alibaba's Qwen3.8-Max, and GLM-5.3—have closed the performance gap with Ope…
Researchers at security firm Adversa discovered that Grok can be tricked into stealing user data—including nam…
OpenAI's Astra model recently solved 10 longstanding problems in mathematics and theoretical computer science—…
Apple researchers introduced LINK, a method that improves cross-lingual knowledge transfer by randomly replaci…
The AI news that matters, in one minute each morning.
Sign up free