Title: Handling Double Descent and Predictive Uncertainty in Bayesian Random Features Subject to Contamination

Abstract:

This study introduces a resilient Bayesian framework for random feature (RF) regression that explicitly addresses potential misspecifications in both the prior and the likelihood through Huber-style contamination sets. Building upon the well-established link between ridge-regularized RF training and Bayesian inference utilizing Gaussian priors and likelihoods, we substitute the standard single prior and likelihood with $\epsilon$- and $\eta$-contaminated credal sets. Inference is then conducted via pessimistic generalized Bayesian updating. We derive explicit, computationally tractable bounds for the resulting lower and upper posterior predictive densities. Our analysis reveals that, under conditions of moderate contamination, ambiguity in the prior and likelihood functions essentially translates into direct contamination of the posterior predictive distribution, generating uncertainty envelopes surrounding the traditional Gaussian predictive. To facilitate robust uncertainty quantification, we propose an Imprecise Highest Density Region (IHDR), demonstrating that it can be efficiently approximated by a modified Gaussian credible interval. Additionally, we establish bounds for the predictive variance—assuming a mild truncation approximation for the upper bound—and prove that these bounds maintain the leading-order proportional-growth asymptotic behavior characteristic of RF models. Collectively, these findings formulate a robustness theory for Bayesian random features, ensuring that predictive uncertainty remains computationally feasible, retains the classical double-descent phase structure, and benefits from explicit worst-case guarantees when prior and likelihood misspecifications are bounded.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC