Title: Replacing Human Curation with Synthetic Augmentation in RLVR

Abstract: The scarcity of high-quality training tasks represents a primary obstacle to applying reinforcement learning from verifiable rewards (RLVR) to agentic language models. Creating each task involves establishing a sandboxed environment, designing a prompt, and manually authoring a reward function; furthermore, only tasks meeting a specific quality threshold yield meaningful training signals. Economic constraints prevent hand-curation from scaling to the volume of tasks required for effective RL training, and the exchange rate between automatically generated task variants and those created by humans remains undefined. This study explores the use of pre-specified, gate-filtered augmentations derived from a limited set of human-authored base tasks as a replacement for additional manual curation during the RLVR process. We define the cost-adjusted trade rate, denoted as $\rho_{\text{cost}}$, which quantifies the value of augmented versus human-authored tasks. By conducting controlled ablations across training datasets with different levels of augmentation, we measure this rate and analyze the overall economics of the augmentation pipeline. Our findings indicate that replacing human-authored tasks with augmented content preserves aggregate generalization performance on a ten-benchmark suite covering code generation, instruction following, reasoning, and multi-turn agentic function calling. Across a realistic range of cost ratios ($c_{\text{human}}/c_{\text{aug}}$), the cost-adjusted trade rate $\rho_{\text{cost}}$ for gated synthetic tasks relative to human-authored ones ranges between $1.4\times$ and $11.6\times$.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC