Title: Unified and Data-Efficient Image-to-Image Translation via Decoupled Residual Denoising Diffusion Models

Abstract:

This study introduces Decoupled Residual Denoising Diffusion models (DRDD), a novel framework designed to achieve both unification and data efficiency in image-to-image (I2I) translation. Although diffusion models have significantly enhanced the quality and diversity of I2I tasks, this work highlights a previously overlooked characteristic of these models. Specifically, beyond their standard function of manifold lifting—shifting data away from low-dimensional manifolds—the introduction of Gaussian noise serves to harmonize domains by implicitly aligning feature distributions across different domains. This property proves especially beneficial for unified I2I translation.

However, conventional diffusion models tend to diminish this harmonization benefit too early because noise removal and residual correction occur simultaneously within a single, coupled diffusion process. To overcome this limitation, DRDD separates the diffusion procedure into two distinct, sequential stages: (1) a stochastic noise diffusion phase dedicated to domain harmonization and manifold lifting, and (2) a deterministic residual diffusion phase that captures the essential semantic mapping entirely within a fixed-noise domain. By decoupling these processes, the model maintains harmonization and manifold lifting effects throughout the entire transformation, which significantly reduces the complexity of learning unified mappings across various domains and tasks.

Importantly, the noise diffusion stage is trained solely on abundant, unpaired images from the target domain, thereby enhancing data efficiency. Both theoretical and empirical evaluations confirm that DRDD integrates seamlessly with mainstream diffusion architectures and consistently provides robust, unified I2I translation capabilities, even when paired data is scarce. The implementation code is publicly accessible at https://github.com/HKU-HealthAI/DRDD.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC