Title: IntraShuffler: A Privacy-Preserving Framework for Heterogeneous Differential Privacy Federated Learning

Abstract:

Heterogeneous Differential Privacy (HDP) within Federated Learning (FL) enables individual clients to tailor their privacy budgets ($\varepsilon_i$) based on specific data sensitivity levels and institutional mandates. To enhance model performance, many HDP-FL implementations utilize $\varepsilon$-aware server aggregation, which re-weights client contributions according to their stated privacy parameters. However, FL gradient updates inherently carry structural signatures resulting from non-independent and identically-distributed (non-IID) data. The $\varepsilon$-aware aggregation process further exposes these signals, creating vulnerabilities that an honest-but-curious server can exploit for inference.

In this study, we demonstrate that a server leveraging gradient denoising and surrogate modeling can execute a Privacy Inference Attack. Under realistic knowledge constraints, this attack successfully identifies client distributional attributes and links updates from the same participant across different training rounds, as evidenced by surrogate inference accuracy and linkage success rates. While the Shuffle-Model is a well-known countermeasure that mitigates such risks by anonymizing update sources, it is fundamentally incompatible with the $\varepsilon$-aware aggregation required by HDP-FL.

To resolve this conflict, we introduce IntraShuffler, a middleware defense framework specifically designed for HDP-FL environments. IntraShuffler employs a privacy-aware shuffling strategy that categorizes clients into buckets based on compatible privacy levels. It then performs parameter-level shuffling within each bucket to break persistent gradient structures without compromising $\varepsilon$-aware aggregation. Our experiments, conducted across four distinct datasets, indicate that IntraShuffler cuts gradient recoverability by more than 60% and lowers surrogate inference accuracy from 0.78 to 0.33, all while sustaining comparable model utility across various FL aggregation methods.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC