Title: Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs

Abstract:

Effective reasoning in large language models hinges not merely on the acquisition of knowledge, but critically on the strategic deployment of cognitive behaviors during the generation process. Current approaches typically depend on explicit control mechanisms at the behavior level, which often lack the adaptability needed to handle the diverse failures and corrective demands that arise across different reasoning states, tasks, and model architectures. To address this limitation, we introduce Latent Reward Steering (LRS), an adaptive framework designed for the inference stage. LRS fosters cognitive behaviors by optimizing the latent states of sparse autoencoders (SAEs), which implicitly encode these behaviors. Instead of utilizing predefined cognitive schemas or static steering vectors, LRS employs a latent reward model trained on reasoning trajectories based on the correctness of final answers. This model evaluates the quality of intermediate latent states. During the inference phase, gradients derived from the reward signal generate correction directions tailored to specific latent states identified as fragile. Furthermore, a dual gate mechanism, incorporating both reward and confidence metrics, ensures that interventions are strictly limited to states flagged as unstable by the reward signal. Empirical evaluations across various reasoning LLM backbones and benchmarks demonstrate that our method consistently outperforms existing baselines. Post-hoc analyses further reveal that LRS implicitly enhances beneficial cognitive behaviors, effectively rectifying initial reasoning errors. The source code is publicly available at: https://github.com/jiakanglee/Latent-Reward-Steering.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC