Title: AgentCL: Establishing Rigorous Standards for Continual Learning Evaluation in Language Agents

Abstract:

While language agents dedicate significant computational resources to resolving specific tasks, the knowledge gained during a single episode is frequently wasted and not leveraged in subsequent interactions. Continual learning posits that agents should gather reusable insights across a sequence of tasks, demonstrating continuous improvement while preventing interference from unrelated data. However, current benchmarks fall short of providing a rigorous assessment of this capability for language agents. Existing studies predominantly concentrate on retrieving and reasoning through extensive conversations or documents, whereas recent benchmarks for lifelong adaptation typically employ simplistic task streams with insufficient scrutiny of inter-task dependencies. This complexity obscures an understanding of how agents learn and recycle information over time.

To address these gaps, this study introduces AgentCL, an evaluation framework designed specifically for continual learning in agents. The framework emphasizes controlled task streams and metrics that measure transfer improvements. AgentCL generates compositional streams where earlier solutions, evidence, or processes are deliberately structured to be reusable in future tasks, contrasting these with naive streams where such reusability is not assured. We utilize this benchmark to assess non-parametric memory architectures for continual learning.

Furthermore, to investigate how specific memory design choices impact continual learning, we developed MemProbe. This probing technique captures interactions, insights, and skills while filtering out unreliable data during the consolidation phase. Our empirical analysis, spanning coding, deep research, and language understanding/reasoning tasks, reveals that naive streams lack the sensitivity to differentiate between various memory designs. In contrast, controlled streams effectively highlight differences in plasticity. Additionally, we observe that naive and held-out settings often result in minimal gains and may even reveal degradation caused by memory issues. These findings underscore the necessity for advanced memory designs that successfully balance the ability to learn new information (plasticity) with the stable retention of reusable knowledge.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC