Title: Advancing Unified Context Evolution for LLM Agents

Abstract:

While Large Language Model (LLM) agents are capable of tackling complex, multi-step interactive tasks by merging reasoning capabilities with environmental feedback, they currently face a significant limitation: every new episode begins with an identical, static context. Consequently, any effective strategies identified during a task are discarded once the episode concludes. Current methodologies typically restrict learning to individual tasks or aggregate all experiences into a single, unstructured repository. These existing systems fail to categorize knowledge types, monitor performance metrics through usage, or identify gaps in their knowledge bases.

To address these challenges, we present Unified Context Evolution (UCE), a gradient-free framework designed to externalize agent experience into a dynamic library composed of typed Evolvable Context Units (ECUs). This approach segments experience into four distinct categories: Memory, Strategy, Workflow, and Skill. Each category is generated from specific trajectories based on tailored conditions. At the moment of decision-making, relevant units are retrieved, their value is assessed through repeated usage outcomes, and obsolete units are pruned. Furthermore, a scheduling mechanism directs the generation budget for each cycle toward the types where the library is most deficient.

In evaluations across two interactive benchmarks, UCE demonstrated substantial improvements, increasing ALFWorld success rates from 75.4% to 96.3% and boosting WebShop task scores from 45.1% to 61.3%. Additionally, the accumulated library proves transferable to alternative actor backbones without the need for retraining.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC