Title: Enhancing Federated Learning Through Variational Bayesian Inference: Achieving Personalization, Sparsity, and Clustering

Abstract:

Federated learning (FL) offers a robust framework for distributed machine learning that safeguards client privacy. Nevertheless, its performance often deteriorates when dealing with data that is both scarce and highly heterogeneous. To address these challenges, we introduce pFedBayes, a novel personalized Bayesian FL methodology. In this approach, each client utilizes the global distribution trained by the server as a prior, subsequently refining their local distribution by minimizing a composite objective function. This function balances the reconstruction error on personalized data against the Kullback-Leibler (KL) divergence from the downloaded global distribution.

To further improve inference efficiency, we propose sFedBayes, a sparse personalized Bayesian FL technique. Additionally, to manage extreme heterogeneity inherent in non-i.i.d. datasets, we develop cFedBayes, a clustered Bayesian FL model that learns distinct prior distributions for various client groups. Our theoretical analysis establishes generalization error bounds for all three methods, demonstrating that their error rates attain minimax optimality within a logarithmic factor. Notably, while pFedBayes provides a single uniform bound, cFedBayes achieves a more refined cluster-level generalization error bound. Extensive empirical evaluations confirm that these proposed methods outperform other state-of-the-art personalized techniques, particularly in scenarios characterized by limited and heterogeneous data.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC