Title: Mastering Diffusion Model Sampling Through Inverse Reinforcement Learning

Abstract: Diffusion models produce samples via an iterative denoising mechanism steered by a pre-trained neural network. Once the denoiser architecture is established, the sampling algorithm—encompassing noise schedules, guidance scales, and stochasticity profiles—demands meticulous tuning. This optimization is traditionally achieved through expensive empirical grid searches. Here, we present an inverse reinforcement learning framework designed to acquire sampling strategies without necessitating denoiser retraining. We model the diffusion sampling process as a discrete-time, finite-horizon Markov Decision Process, in which actions represent potential adjustments to the sampling dynamics. Rather than specifying an explicit reward function, we optimize action scheduling by directly aligning with the target behavior anticipated from the sampler, employing policy gradient methods. Our experimental results demonstrate that this method achieves performance comparable to fine-tuned samplers while incurring significantly lower expenses than grid search. Specifically, on ImageNet-64, one training iteration supersedes exhaustive search with costs up to nine times lower, introducing merely a 16% overhead during inference.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC