Title: Bridging CoCoA and ADMM in Distributed Empirical Risk Minimization

Abstract: Distributed empirical risk minimization (ERM) is typically analyzed through two distinct methodological families: CoCoA algorithms, which originate from distributed dual coordinate ascent, and ADMM algorithms, which stem from consensus and proximal splitting techniques. Although these approaches appear separate, this study explores their interrelation through a unified primal-dual framework. We demonstrate that several key algorithms—including consensus ADMM, linearized consensus ADMM, two variants of distributed proximal ADMM, and ridge-regularized CoCoA—can all be expressed using a shared update structure that incorporates a global primal variable alongside block dual variables.

This reformulation reveals previously obscured connections. Specifically, for ridge-regularized ERM, CoCoA is shown to be identical to a specific proximal ADMM scheme when viewed through the lens of dual updates. Furthermore, under an explicit parameter mapping and a sign reversal of the saddle objective, primal consensus ADMM is equivalent to dual proximal ADMM; analogous relationships exist for their linearized counterparts. These findings suggest that, for ridge-regularized ERM problems, ADMM-type algorithms, when appropriately optimized, perform at least as well as CoCoA. Additionally, this unified perspective provides a natural primal-dual gap stopping criterion for consensus ADMM and enables a consolidated $O(1/T)$ ergodic convergence analysis for ADMM-based methods. Experimental results on synthetic regression tasks and real-world SVM datasets corroborate these theoretical relationships, elucidate the impact of tuning parameters, and demonstrate that properly calibrated ADMM variants can surpass CoCoA in ridge-regularized scenarios.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC