Title: Achieving Near-Optimal Decentralized Stochastic Convex Optimization Across Networks

Abstract

This paper investigates decentralized stochastic smooth convex optimization, a scenario in which $M$ workers seek to minimize an average objective function. The process relies on local stochastic gradients and communication restricted to immediate neighbors within a static gossip network. A pivotal inquiry in this domain involves identifying the maximum number of participants that can be accommodated under a total gradient sample budget of $N$, while still maintaining the statistical convergence rate of $O(1/\sqrt N)$ characteristic of centralized methods.

We propose an accelerated decentralized algorithm that sustains this convergence rate for a network size of up to $M\lesssim \sqrt{\rho}\,N^{3/4}$, where $\rho$ denotes the spectral gap of the gossip network. This advancement significantly improves upon the previous best-known maximal scaling limit of $M\lesssim \rho\sqrt N$. The proposed approach leverages a one-step-delayed stochastic acceleration mechanism, allowing workers to combine minibatching with accelerated gossip protocols while effectively managing residual disagreement. Furthermore, the method’s performance guarantee exhibits only logarithmic dependence on the heterogeneity between local objectives.

To validate the efficiency of our approach, we derive a corresponding lower bound for linear-span decentralized first-order methods. This result demonstrates that our method is optimal up to logarithmic factors.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC