Title: Language Models Require Rest: Mechanisms for Self-Modification and Memory Consolidation

Recent years have seen a dramatic evolution in machine learning algorithm design, transitioning from early, narrow shallow models to the broad capabilities of modern Deep Large Language Models (LLMs). While these current systems demonstrate impressive proficiency in tasks requiring immediate prediction or in-context learning, they remain limited by their inability to engage in continuous learning or to effectively transfer temporal, in-context insights into stable long-term parameters.

Drawing inspiration from human cognitive processes, this study proposes a "Sleep" paradigm designed to enable models to learn continuously. This framework facilitates the distillation of transient, short-term memories into robust, long-term knowledge through the use of replay mechanisms, while also allowing for recursive self-improvement via a "Dreaming" phase.

The sleep process is structured into two distinct stages:

1. Memory Consolidation: This involves an upward distillation method termed "Knowledge Seeding." Here, the memories held by a smaller model are distilled into a larger network, thereby increasing capacity without sacrificing existing knowledge. As a proof of concept, the authors introduce a novel Generalized Distillation process for Knowledge Seeding, which integrates on-policy distillation with Reinforcement Learning (RL)-based imitation learning.
2. Dreaming: This stage serves as a self-improvement mechanism. Using RL, the model generates a curriculum of synthetic data to rehearse new information and refine established skills, all without the need for human supervision.

Experimental results across tasks involving long-horizon planning, continual learning, knowledge integration, and few-shot generalization underscore the critical value of incorporating this sleep stage.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC