Title: Enhancing Visual Representation Alignment Generation via GRPO

Abstract: Although recent diffusion transformers have shown impressive prowess in image synthesis, their training processes remain inefficient, largely due to a misalignment between generative and discriminative representations. While frameworks like REPA facilitate convergence by aligning noisy denoising features with pretrained visual encoders, their alignment loss is externally supervised, static, and lacks the adaptability needed during both training and inference. Current approaches typically depend on fixed cosine alignment or contrastive objectives. These static methods fail to dynamically balance representation consistency with generation quality, leading to limited discriminative advantages and an inability to optimize alignment in a task-adaptive way.

To overcome these limitations, we introduce VRPO, a reinforcement-based optimization strategy that substitutes REPA’s static alignment loss with an objective focused on generative representation policy optimization. Rather than imposing a rigid similarity constraint, VRPO conceptualizes representation alignment as a reward-guided mechanism. The model is assigned adaptive rewards determined by generation fidelity, perceptual quality, and the semantic coherence between diffusion features and pretrained visual embeddings. This approach allows the generator to iteratively refine its internal representations toward semantically significant directions, thereby boosting image quality.

VRPO integrates smoothly into diffusion transformers, adding minimal computational overhead while maintaining full compatibility with SiT and DiT architectures. Comprehensive experiments conducted on ImageNet-256x256 reveal that VRPO-Alignment significantly improves both convergence rates and fidelity. Under identical compute budgets, it delivers up to a 1.8 FID improvement and accelerates training speed by 2.3 times compared to REPA.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC