Title: The Impact of Neural Losses on VAE Latent Structures

Original: arXiv:2606.00635v1 Announce Type: new Abstract: While the standard $\beta$-VAE objective relies on pointwise likelihood, contemporary Variational Autoencoders (VAEs) are seldom trained using this metric in isolation. Instead, practitioners typically integrate perceptual and adversarial losses alongside pointwise reconstruction, even though the implications of these combinations for the model's latent dynamics remain poorly understood. This study demonstrates that the selection of reconstruction loss fundamentally alters the rate-distortion framework, modifying both the geometric structure and information capacity of the latent space in ways that cannot be detected by examining reconstructions alone. We provide both theoretical proofs and empirical evidence showing that incorporating neural components—such as adversarial and perceptual objectives—diminishes the quantity of information encoded within the latent representations. Furthermore, our analysis reveals that neural reconstruction losses induce systematic geometric shifts, rendering the latent space more isotropic and spreading uncertainty more uniformly across dimensions, which results in distinct posterior variance patterns. These results underscore the limitations of the rate-distortion tradeoff as a sole explanatory framework for VAE behavior. Consequently, we advocate for a more mechanistic perspective to explore how varying distortion metrics transform the underlying optimization landscape.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC