Title: Broadening the Search: Coordinated Pass@K Policy Optimization for Code Reasoning

Abstract: Utilizing a verifier to sample repeatedly is the conventional approach for distributing test-time computational resources in code generation, typically measured by the pass@$K$ metric. However, traditional policy classes generate $K$ independent samples from a single distribution, which frequently results in near-duplicate reasoning paths and inefficient use of the computational budget due to redundant rollouts. This inefficiency is particularly problematic in competitive programming, where problems often have multiple distinct algorithmic solutions, yet pass@$K$ is satisfied by a single correct attempt. To address this, we introduce Coordinated Pass@$K$ Policy Optimization (CPPO), which reframes pass@$K$ generation as joint strategy exploration. In this framework, a planner proposes a tuple of $K{=}4$ alternative high-level methods, while a shared solver attempts to solve the problem using one solution per method. CPPO trains this joint policy using a multiplicative planner reward, defined as $R_{\mathrm{plan}} = J_\psi \cdot R_{\mathrm{out}}$, which grants credit exclusively to valid strategy tuples that achieve verifier-confirmed pass@$K$ success. Evaluated across APPS, CodeContests, and LiveCodeBench-v6, CPPO demonstrates superior pass@$4$ performance compared to direct sampling, planning baselines, planner-only supervised fine-tuning (SFT), and pass@$K$-oriented reinforcement learning, all under an identical $K{=}4$ solver-attempt budget. The method yields statistically significant improvements in six out of nine model-benchmark combinations. Notably, the most substantial single improvement was observed on Qwen3.5-9B in LiveCodeBench-v6, where CPPO outperformed the strongest baseline, PKPO, raising scores from 0.588 to 0.748 (paired bootstrap, $p < 0.05$).

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC