Title: Beyond the Frontier: Stochastic Backtracking for Efficient Test-Time Scaling

Abstract

Test-time scaling enhances the reasoning capabilities of language models by allocating additional computational resources to explore multiple solution paths. The primary objective is to optimize accuracy while reducing the total volume of tokens generated during the reasoning process. While recent methods guided by Process Reward Models (PRMs) utilize intermediate prefix scores to direct this search, the majority operate on a "frontier-only" basis. These approaches retain only the currently active prefixes and irreversibly discard or resample others based on noisy PRM evaluations. Such strategies risk premature commitment, a collapse in diversity, and the elimination of prefixes that may still lead to correct solutions.

To address these limitations, we introduce stochastic backtracking over a persistent pool of historical prefixes. This approach enables test-time computation to revisit previously generated states rather than focusing exclusively on the current frontier. We propose two complementary mechanisms to ensure efficiency. First, Subpool Selection enhances greedy PRM-guided search by implementing Top-N selection within random subpools, thereby allowing historical prefixes to bypass frontier candidates that may have been over-scored. Second, Power Backtrack Sequential Monte Carlo adapts SMC-style resampling to the persistent pool, utilizing powered PRM scores and mixture-corrected weights.

Evaluated across various model scales and mathematical reasoning benchmarks, our methods consistently deliver higher accuracy per token compared to strong PRM-guided baselines. Furthermore, they achieve equivalent accuracy levels using significantly fewer tokens. These results demonstrate that persistent-pool stochastic backtracking offers a straightforward and effective strategy for improving the accuracy-token trade-off in test-time scaling.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC