Title: Variance-Gated Ensembles: An Epistemic-Aware Framework for Uncertainty Estimation

Abstract:

In machine learning, the ability to rapidly and accurately estimate per-sample uncertainty is essential. Traditionally, practitioners have relied on predictive distributions generated by Bayesian techniques or their approximations, relying on an additive decomposition of uncertainty into aleatoric (data-driven) and epistemic (model-driven) components. However, recent studies have challenged the validity of this additive approach, highlighting that it fails under conditions involving finite-ensemble sampling or when predictive distributions are mismatched.

To address these limitations, this paper proposes Variance-Gated Ensembles (VGE), a distinct and differentiable framework designed to incorporate epistemic sensitivity through a signal-to-noise gate derived from ensemble statistics. The VGE architecture offers two primary innovations: first, a Variance-Gated Margin Uncertainty (VGMU) metric that integrates decision margins with the predictive variance of the ensemble; and second, a Variance-Gated Normalization (VGN) layer. This layer facilitates the integration of variance-gated uncertainty mechanisms into the training process by applying learnable, per-class normalization to the probabilities of individual ensemble members.

By deriving closed-form vector-Jacobian products, the framework allows for end-to-end training that accounts for the sample mean and variance of the ensemble. Experimental results demonstrate that VGE performs on par with or better than current state-of-the-art information-theoretic baselines, all while maintaining high computational efficiency. Consequently, VGE presents a robust, scalable, and practical solution for epistemic-aware uncertainty estimation within ensemble models.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC