Title: MemTrain: Enhancing LLM Context Memory via Self-Supervised Training

Abstract:

For long-horizon Large Language Model (LLM) agents, memory is a critical function that allows them to retain and leverage information gathered over prolonged interactions. Current methods for training memory-aware agents usually rely on end-to-end reinforcement learning applied directly to downstream tasks. However, this approach faces significant hurdles: acquiring high-quality, annotated data for complex memory scenarios is expensive, and the resulting datasets often lack the diversity needed to capture a broad spectrum of general memory behaviors.

To address these limitations, we introduce MemTrain, a self-supervised framework designed to broadly strengthen the context-memory capabilities of LLM agents, thereby facilitating more effective post-training for specific tasks. MemTrain leverages unlabeled Wikipedia datasets to construct two coupled proxy tasks. The first is an end-to-end masked reconstruction objective, which challenges the model to recover masked entities following multiple rounds of memory updates. This mechanism promotes the maintenance of memory from the perspective of the final output. The second is an intermediate memory recall objective, requiring the model to reconstruct masked historical data using intermediate memory states. This encourages faithful compression and ensures memory completeness throughout the interaction. Both objectives are optimized jointly using GRPO.

Our extensive evaluations on long-text question answering and search-based QA benchmarks reveal that MemTrain consistently boosts memory-intensive reasoning performance across various models. Notably, it achieves improvements of up to 17.67 points compared to direct task-specific post-training.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC