Title: Latent Reasoning in TRMs is Secretly a Policy Improvement Operator

Abstract:

Small-scale models employing latent recursion have recently demonstrated significant promise in tackling complex reasoning challenges. The prevailing explanation for this success posits that recursion effectively deepens the network, enabling it to mimic the capabilities of much larger architectures with greater compactness. However, models utilizing recursive layers consistently underperform compared to non-recursive counterparts possessing the same feed-forward depth. This discrepancy suggests that not every iteration within the recursive loop meaningfully contributes to increasing the model’s depth, prompting the critical inquiry: under what conditions does latent reasoning enhance performance, and when does it merely generate wasted computational effort?

Our research addresses this question by demonstrating that latent recursive reasoning can be rigorously formalized as a policy improvement algorithm. Leveraging these findings, we introduce novel training methodologies adapted from reinforcement learning and diffusion techniques specifically for latent reasoning models. Employing the Tiny Recursive Model as our experimental framework, we show that these adjustments eliminate ineffective computational steps, cutting the total number of forward passes by a factor of 18 without compromising accuracy. Ultimately, our work illustrates how viewing recursive steps through the lens of policy improvement not only clarifies model dynamics but also offers a pathway for future architectural enhancements.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC