Title: RichControl: Structure- and Appearance-Rich Training-Free Spatial Control for Text-to-Image Generation

Abstract:

Text-to-image (T2I) diffusion models have achieved significant success in producing high-fidelity images from textual descriptions. To enable more precise spatial control, recent advancements have integrated conditional inputs, such as canny edges, into these models. Within this domain, feature injection techniques have gained traction as a viable, training-free substitute for conventional fine-tuning strategies. Nevertheless, these methods frequently encounter challenges like structural misalignment, condition leakage, and visual artifacts, particularly when the reference conditions deviate substantially from standard RGB image distributions.

Our analysis of current approaches reveals a critical oversight: the sampling schedule for condition features, which has largely been unexplored, does not adequately address the dynamic relationship between maintaining structural integrity and aligning with the target domain across different diffusion steps. Leveraging this insight, we introduce a flexible, training-free framework that separates the sampling schedule of condition features from the denoising trajectory. We conduct a systematic investigation into various feature injection schedules to strike an optimal balance between structural fidelity and aesthetic quality.

Furthermore, we refine the sampling process by implementing a restart refinement schedule and boost visual output through an appearance-enhancing prompting strategy. These innovations collectively facilitate controllable generation that is both structurally robust and visually rich without requiring training. Comprehensive experiments confirm that our approach delivers state-of-the-art results across complex and varied conditions. Due to its broad applicability, the framework seamlessly supports compositional conditional generation and functions in a plug-and-play capacity across diverse architectures, ranging from UNet-based diffusion models to contemporary DiT backbones like FLUX.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC