Title: Robust Performance in Inverse Problems Despite Limited Diffusion Priors

Abstract: Is it possible for a diffusion model trained exclusively on bedroom scenes to effectively reconstruct human faces? While diffusion models are frequently employed as priors to solve inverse problems, conventional wisdom suggests that these models must be high-fidelity and trained on data closely aligned with the target signal. However, real-world scenarios often necessitate the use of mismatched or lower-quality diffusion priors. Contrary to expectations, such "weak" priors frequently achieve performance levels comparable to robust, in-domain baselines. This paper investigates the specific conditions under which inverse solvers maintain robustness against these weaker priors. Our extensive experimental results indicate that weak priors are effective when measurements provide substantial information, such as when a large number of pixels are observed, and we delineate the specific regimes where they are likely to fail. To elucidate this phenomenon, we integrate Bayesian-consistency theory with local-correlation analysis. The theoretical framework establishes the conditions under which high-dimensional measurements cause the posterior distribution to concentrate around the true signal, while the correlation analysis demonstrates that both weak and strong priors for natural images exhibit similar local spatial structures. These findings offer a rigorous justification for the reliable application of weak diffusion priors. The associated code can be accessed at https://github.com/jjia131/weak-diffusion-priors-inverse-problem.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC