Title: Adaptive Information Control for Search-Augmented LLM Reasoning

Abstract:

Search-augmented reasoning agents combine multi-step logical deduction with external retrieval mechanisms. However, unrestricted retrieval can lead to the accumulation of redundant evidence, context saturation, and instability in reinforcement learning (RL). Current outcome-based RL approaches typically offer only sparse terminal rewards, providing insufficient guidance for intermediate decisions regarding information acquisition. To address this, we introduce DeepControl, an adaptive framework grounded in information utility, which serves as a state-dependent estimate of the marginal value of retrieved evidence. This system manages information acquisition across two dimensions: extent, determining whether retrieval should proceed, and resolution, controlling the volume of retrieved detail presented. These controls are executed via retrieval-continuation guidance, hierarchical granularity management, and an annealed control-forcing mechanism. Consequently, the policy internalizes efficient acquisition strategies during training and functions independently of external control during inference. Evaluated across seven benchmarks, DeepControl consistently surpasses robust RL and retrieval baselines that lack explicit information control. Notably, it yields average performance gains of +9.4 and +8.6 points over Search-R1 on the Qwen2.5-7B and Qwen2.5-3B models, respectively. Further analysis confirms enhancements in search efficacy, training stability, and the utilization of evidence.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC