Title: StepPRM-RTL: Enhancing RTL Synthesis via Stepwise Process-Reward Guided Fine-Tuning of LLMs

Abstract

The automatic generation of Verilog and VHDL code for digital hardware designs continues to face significant hurdles, primarily driven by the need for long-horizon reasoning, complex multi-step dependencies, and rigorous correctness requirements. To address these challenges, we introduce StepPRM-RTL, an innovative framework that integrates retrieval-augmented fine-tuning (RAFT), process-reward modeling (PRM), and stepwise trajectory modeling. This approach is designed to boost both the reasoning fidelity and functional accuracy of large language models (LLMs) tasked with RTL code generation.

StepPRM-RTL builds stepwise reasoning trajectories derived from canonical solutions, ensuring that each step includes both a logical rationale and an incremental code modification. A Process Reward Model (PRM) assesses these intermediate steps, offering dense feedback that steers reinforcement-style updates during the RAFT fine-tuning process. Furthermore, Monte Carlo Tree Search (MCTS) is employed to investigate alternative reasoning paths, thereby enriching the training dataset with high-quality trajectories. By combining stepwise and outcome-aware rewards, the model acquires the ability to understand not just how to build correct RTL, but also why, thereby advancing long-horizon reasoning capabilities beyond the limits of standard supervised or outcome-based training methods.

Experimental results on benchmark Verilog and VHDL datasets indicate that StepPRM-RTL surpasses the most effective existing methods by more than 10% in metrics related to functional correctness and reasoning fidelity. Ablation studies highlight that the synergy between PRM-guided rewards and stepwise trajectory exploration is critical to this performance boost. StepPRM-RTL demonstrates versatility across different RTL languages and offers a scalable framework for producing high-fidelity, interpretable code, setting a new benchmark for LLM-assisted hardware design automation.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC