Title: Do Heavy Tails Assist Diffusion? Navigating the Delicate Balance Between Initialization and Training

Abstract: Recent literature has suggested integrating heavy-tailed (HT) noise into diffusion and flow-based generative architectures, aiming to enhance the recovery of distribution tails and boost generative diversity. The rationale is straightforward: for data exhibiting heavy tails, HT noise may seem more appropriately matched than traditional light-tailed (LT) Gaussian noise. However, substituting Gaussian noise with HT noise fundamentally alters the underlying estimation task. In this study, we re-examine this approach through a comprehensive theoretical and empirical analysis, deriving sampling-error bounds for two prominent diffusion models utilizing HT and LT noise, respectively. Our analysis reveals that HT noise complicates the statistical estimation process, resulting in inferior sampling-error bounds. We validate these theoretical insights through experiments on both synthetic and real-world datasets, successfully demonstrating the predicted error trade-off. Consequently, our findings cast doubt on the increasing preference for HT noise in generative model design and question its efficacy in facilitating exploration of rare regions.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC