Title: Decentralized EM Algorithm for Gaussian Mixtures under Data Heterogeneity and Partial Labeling

Abstract: This paper provides a comprehensive examination of various network-based Expectation-Maximization (EM) algorithms applied to the Gaussian mixture model within the framework of decentralized federated learning (DFL). Our theoretical analysis reveals that a straightforward application of the traditional EM algorithm in DFL results in a biased estimator when data distributions are heterogeneous across different nodes. To mitigate this bias, we propose the momentum network EM (MNEM) algorithm, which incorporates data from both current estimates and historical estimators derived from prior DFL iterations. Additionally, we present a semi-supervised variant, termed semi-MNEM, designed to leverage insights from partially labeled datasets. Theoretical rigor confirms that, subject to standard regularity conditions, the MNEM estimator attains asymptotic efficiency comparable to that of a whole-sample estimator, even in the presence of data heterogeneity. Furthermore, the semi-MNEM approach markedly accelerates the convergence rate of the MNEM algorithm, particularly in scenarios where mixture components exhibit poor separation. We validate the finite-sample performance of these proposed methods through extensive simulations and an analysis of a standard chest X-ray dataset.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC