Title: Fixed-Mean Gaussian Processes for Post-hoc Bayesian Deep Learning

Abstract: Post-hoc uncertainty estimation for pre-trained deep neural networks (DNNs) has recently garnered significant attention. These techniques augment existing back-propagation-trained models with confidence metrics, such as error bars, to quantify prediction uncertainty while maintaining the network's original accuracy. In this work, we propose a new class of sparse variational Gaussian processes (GPs) characterized by a posterior mean that can be constrained to any continuous function, provided a universal kernel is employed. By anchoring the GP’s mean to the outputs of the pre-trained DNN, our method effectively utilizes the GP’s predictive variances to capture the uncertainty of the DNN’s predictions. We employ variational inference (VI) to facilitate efficient stochastic optimization, ensuring that training costs do not scale with the volume of training data. This efficiency allows the approach to handle large-scale datasets like ImageNet. Named the Fixed-Mean GP (FMGP), this architecture-agnostic framework depends exclusively on the pre-trained model’s outputs to refine predictive variances. Our experiments indicate that FMGP surpasses current state-of-the-art techniques in both computational efficiency and the quality of uncertainty estimation for post-hoc Bayesian inference in DNNs.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC