Title: Investigating the Cross-Scenario Generalization of Agentic Memory Systems: Diagnostics and a Robust Baseline

Abstract: As Large Language Model (LLM) agents accumulate interaction histories that exceed their available context windows, there is increasing academic interest in memory systems. However, the majority of current designs are optimized for specific use cases, such as multi-session conversations or singular trajectory formats, with scant evidence demonstrating their ability to generalize across the diverse trajectories agents face in real-world deployments. This study re-evaluates eight existing memory systems alongside an agentic harness designed for search problems, testing them across five distinct scenarios: single-turn question answering, multi-session chat, agentic-trajectory question answering, memory stress tests, and long-horizon agentic tasks. Our proposed harness, which utilizes tool calls to autonomously manage flat text-file storage, secured the highest ranking in cross-task performance. These results indicate that memory efficacy relies more on granting agents active control over storage and retrieval mechanisms than on relying on passive storage architectures within fixed pipelines. We translate this finding into AutoMEM, an agentic memory harness featuring a self-managed tool interface, which demonstrates superior cross-scenario generalization compared to the other systems assessed in our evaluation.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC