
What happened
Researchers from Google and Cornell introduced a "sleep framework" where an LLM alternates active interaction with an offline phase that consolidates short-term context into long-term parameters, retaining knowledge up to 10M tokens.
Why it matters
This approach enables deployed models to keep learning new facts and improve reasoning after release, rather than staying frozen at a fixed knowledge cutoff, while keeping the core architecture unchanged.
WHO IT HITSEnterprise teams running self-hosted LLMs could adopt this periodic consolidation cycle to keep models current with new data without full retraining, though the framework is still research-stage as described.
Summaries like this, every morning.
The sleep framework addresses a fundamental limitation of deployed LLMs: once trained, they operate with a fixed knowledge cutoff and cannot acquire new capabilities without risking catastrophic forgetting. Traditional continual learning updates often degrade performance on previously learned tasks, while in-context learning only retains information within a single session. By mimicking the brain's rest cycles, the framework introduces a periodic offline phase where the model consolidates short-term context into long-term parameters. This is achieved through parameter expansion using Mixture-of-Experts and knowledge seeding via on-policy distillation and reinforcement learning-based imitation. During the "dreaming" stage, the model generates synthetic training samples, filters them for usefulness, and refines its capabilities. Empirical results show the approach maintains near-zero performance drop in continual translation for unseen languages and achieves strong scores on reasoning benchmarks like AIME 24/25 and HMMT 25. The theoretical foundation suggests that consolidation acts as low-rank compression, while dreaming diversifies gradient directions for more stable optimization. This research points toward AI systems that can continually learn and self-improve after deployment, though practical adoption would require further validation.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
With Google Chrome's Modern Web Guidance plugin loaded, Claude Opus 5.5 wrote 2,740 JavaScript lines across 16…

A paper by University of Macau and The Hong Kong Polytechnic University researchers tested web-application dev…

Google detailed the Mirror Dimension, an isolated replica internet with social media, shopping and bank dashbo…

On IPO day, Dario Amodei told the Nasdaq opening bell crowd Anthropic will pace training and deployment of its…

Meta employees said Muse mixed up chats across accounts and surfaced messages and bank balances they never sha…

Anthropic's shares began trading on the Nasdaq at $111.50, 33% above the $83 IPO price set Wednesday, which wa…
