Title: Enhancing Federated Learning Resilience Through Dimensionality Reduction: Theoretical Insights and Convergence Proofs

Federated Learning (FL) facilitates collaborative model training among distributed clients while preserving data privacy by avoiding the exchange of raw information. However, this architecture remains susceptible to Byzantine attacks. While current robust methods mitigate these threats, they suffer from significant computational burdens during high-dimensional gradient aggregation. As modern models expand in size, this overhead scales poorly and increasingly constitutes the dominant cost of the training process.

To overcome this computational bottleneck, we introduce Projected Dimensionality Reduction (PDR), a universal acceleration framework designed for vector-level distance-based robust aggregators. PDR enhances efficiency by compressing gradients into a significantly smaller subspace using sparse random projection, thereby enabling the rapid calculation of reliability weights. This method optimizes the server’s computational complexity to $\mathcal{O}(Mp)$, where $M$ represents the number of clients and $p$ denotes the model dimension. This complexity matches the theoretical lower bound necessary simply to read the gradients.

We provide convergence guarantees under standard FL assumptions commonly used in prior Byzantine-robust FL studies. By applying the Subspace Embedding Theorem, we demonstrate that PDR attains optimal convergence rates: $\mathcal{O}(1/\sqrt{T})$ for non-convex functions and $\mathcal{O}(1/T)$ for strongly convex functions, with $T$ indicating the number of iterations. Importantly, our mathematical analysis reveals that this substantial acceleration incurs minimal penalty, increasing the inherent Byzantine error floor by only a bounded, tunable factor of $\frac{1+\epsilon}{1-\epsilon}$. Benchmarks on standard datasets validate that combining PDR with existing aggregators delivers speedups of several orders of magnitude in time efficiency, all while preserving highly competitive convergence outcomes.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC