Title: Broad Symmetry Preservation Guarantees in Variational Inference Under Even and Elliptical Constraints

Abstract:

Variational inference (VI) seeks to approximate a target probability density $p$ by identifying the optimal distribution $q$ within a manageable family of models. This optimization is typically achieved by minimizing a divergence measure $D(p||q)$ between the two distributions. Although various divergence metrics have been introduced as objective functions for VI—each yielding distinct approximation outcomes—our analysis demonstrates that these approximations consistently adhere to specific symmetry-matching rules, regardless of which divergence is minimized.

Our findings apply to the extensive category of $f$-divergences, encompassing forward and reverse Kullback-Leibler (KL) divergences as well as $\alpha$-divergences. We establish that if the target distribution $p$ exhibits even symmetry, any stationary point of an $f$-divergence is mathematically guaranteed to correctly identify the mean of $p$. Similarly, when $p$ possesses elliptical symmetry, any stationary point is assured to recover its correlation matrix.

Crucially, these guarantees are derived under the assumption that both $p$ and $q$ are unimodal. Notably, we do not impose stricter conditions such as log-concavity, light-tailed behavior, or universal smoothness. Consequently, these results broaden a prior theorem that was limited to the reverse KL divergence under the assumption of log-concavity for $p$. Furthermore, our framework accommodates scenarios involving partial symmetry, where the target density $p$ displays symmetry along only a subset of its coordinates. Such partial symmetries are commonly encountered in Bayesian hierarchical models, where the prior creates complex geometric challenges while still maintaining specific axes of symmetry.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC