Title: Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill

Abstract:

Reward models (RMs) serve as essential feedback mechanisms for the post-training of large language models (LLMs), playing a pivotal role in reinforced fine-tuning (RFT) and reinforcement learning (RL) workflows. Despite their importance, current reward evaluation methodologies depend on a fragmented array of heterogeneous standards, including rule-based verifiers, ground-truth references, procedural checklists, and intricate rubrics. To date, no unified mechanism has been developed to effectively integrate these varied forms of evidence.

Addressing this gap, we introduce the Skill Reward Model (Skill-RM), a comprehensive framework that reconceptualizes reward modeling as the execution of a reusable "Reward-Evaluation Skill." By framing reward computation as a structured agentic task, Skill-RM establishes a consistent interface designed to orchestrate diverse resources. It dynamically selects and aggregates evidence based on the specific needs of each input, thereby moving beyond static evaluation methods to ensure both consistency and transparency across a wide range of tasks.

Comprehensive experiments conducted on reward benchmarks and downstream applications—such as best-of-N selection and reinforcement learning—reveal that Skill-RM consistently surpasses traditional judge baselines. Our results indicate that Skill-RM offers not only a unified approach to reward modeling but also delivers superior performance through the strategic and dynamic management of evidence.

The code is available at https://github.com/Qwen-Applications/Skill-RM.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC