Title: MulFeRL: Boosting Reinforcement Learning with Verbal Feedback in a Multi-turn Loop

Abstract:

While Reinforcement Learning with Verifiable Rewards (RLVR) is a prevalent method for enhancing reasoning capabilities across various fields, it frequently relies on outcome-only scalar rewards that are often sparse and lack detail. This deficiency is particularly acute when dealing with failed samples; in such cases, scalar rewards merely signal that a solution is wrong without elucidating the specific points where the reasoning process failed. To address this, our study utilizes more comprehensive verbal feedback to steer RLVR on unsuccessful instances, transforming the progress derived from this feedback into trainable learning signals. We introduce MulFeRL (Multi-turn Feedback-guided Reinforcement Learning), a framework that employs a multi-turn, event-triggered approach to RLVR. This system integrates three core components: progress induction to facilitate the regeneration of failed samples based on feedback, progress credit assignment to enable learning from verifier-validated advancements, and structured feedback injection to embed feedback directly into the model’s reasoning workflow. Evaluated on sampled OpenR1-Math data, MulFeRL surpasses supervised, self-distillation-based, and standard RLVR baselines within the same domain, while also demonstrating robust generalization to out-of-domain tasks.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC