AIToday
Large Language ModelsLobsters AIPublished: Nov 19, 2026, 10:00 JST

Google, Cornell sleep framework retains 10M tokens in LLMs

Send on LINE
Google, Cornell sleep framework retains 10M tokens in LLMs

3 Key Points

  1. What happened

    Researchers from Google and Cornell introduced a "sleep framework" where an LLM alternates active interaction with an offline phase that consolidates short-term context into long-term parameters, retaining knowledge up to 10M tokens.

  2. Why it matters

    This approach enables deployed models to keep learning new facts and improve reasoning after release, rather than staying frozen at a fixed knowledge cutoff, while keeping the core architecture unchanged.

WHO IT HITSEnterprise teams running self-hosted LLMs could adopt this periodic consolidation cycle to keep models current with new data without full retraining, though the framework is still research-stage as described.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

The sleep framework addresses a fundamental limitation of deployed LLMs: once trained, they operate with a fixed knowledge cutoff and cannot acquire new capabilities without risking catastrophic forgetting. Traditional continual learning updates often degrade performance on previously learned tasks, while in-context learning only retains information within a single session. By mimicking the brain's rest cycles, the framework introduces a periodic offline phase where the model consolidates short-term context into long-term parameters. This is achieved through parameter expansion using Mixture-of-Experts and knowledge seeding via on-policy distillation and reinforcement learning-based imitation. During the "dreaming" stage, the model generates synthetic training samples, filters them for usefulness, and refines its capabilities. Empirical results show the approach maintains near-zero performance drop in continual translation for unseen languages and achieves strong scores on reasoning benchmarks like AIME 24/25 and HMMT 25. The theoretical foundation suggests that consolidation acts as low-rank compression, while dreaming diversifies gradient directions for more stable optimization. This research points toward AI systems that can continually learn and self-improve after deployment, though practical adoption would require further validation.

FAQ
How does the sleep framework prevent forgetting old knowledge?
It uses parameter expansion with Mixture-of-Experts to add new low-rank experts into stable memory blocks, then resets temporary high-frequency parameters after consolidation.
What evidence supports the framework's performance?
On the BABILong benchmark, it scaled near-perfectly up to 10M tokens, outperforming GPT-4 and Titans. On factual knowledge tests, it scored 48.9% on single-task and 46.2% on CPT evaluations.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articlePalo Alto to pay $4.5 billion for Applied Compute