Title: HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems

Abstract:

As Large Language Model (LLM) agents face increasing demands to operate across diverse task environments requiring distinct execution methods, rigid agent architectures are proving inadequate. This shift necessitates a move toward system-level meta-adaptation that transcends isolated component modifications. Although prior research has focused on either adapting external execution harnesses or training internal reasoning policies, comprehensive system-level adaptation has not been fully explored. Specifically, the adaptation landscape bridging structural execution and reasoning behavior lacks explicit definition, and the synergy between the external harness and the internal reasoner has not been jointly optimized.

To address these gaps, we introduce HarnessForge, a meta-adaptive framework designed to evolve LLM agent systems. This framework conceptualizes an agent system as a coupled "harness--policy" pair, establishing a stable adaptation space that distinctly separates the execution structure at the harness level from the reasoning behavior at the policy level. HarnessForge facilitates the co-evolution of these components through two key mechanisms: fault-guided harness tailoring and harness-conditioned policy alignment.

Our experimental evaluation across five benchmarks spanning various domains demonstrates that HarnessForge consistently enhances the performance of both Qwen3-4B and Qwen3-8B backbones. The framework surpasses baselines that rely solely on harness or policy adjustments, delivering performance gains of up to 12.0\% compared to the strongest existing baseline, while also securing favorable trade-offs in rollout efficiency. These results underscore the efficacy of harness--policy co-evolution and highlight the critical importance of ensuring executable compatibility between the harness and the reasoning policy for successful agent-system adaptation.

The source code for HarnessForge is publicly accessible at https://github.com/mingju-c/HarnessForge.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC