Title: Stability Analysis of Orthogonalized Matrix Momentum Under Client Sampling

Abstract: This paper investigates the finite-sample generalization capabilities of a distributed optimization framework that employs matrix-valued parameters and orthogonalized momentum updates, specifically within a client-sampled environment. Our primary focus is on quantifying the discrepancy between the population and empirical objectives at the final model, particularly when only a fraction of clients are active in each iteration. Assuming independent and heterogeneous client data, varying local sample sizes, and constant aggregation weights, we establish a finite-round upper-tail bound. This result is derived from a stability recursion involving coupled neighbors, augmented by a weighted concentration step. The resulting bound incorporates the client-selection frequencies via an amplification factor, denoted as $Y_i(\mathcal C)$. In scenarios involving uniform full participation and full-batch processing, provided that horizon-dependent amplification terms are appropriately bounded, the bound achieves a scaling of $\widetilde{\mathcal O}(n^{-1}+n^{-1/2})$. The matrix-orthogonalization mechanism must satisfy a Lipschitz condition along paired trajectories; this requirement is met by regularized polar-type mappings and normalized finite-step Newton–Schulz orthogonalizers. Conversely, for the unregularized matrix sign function, the analysis necessitates coupled spectral separation, while Gaussian smoothing facilitates a finite-round smoothed alternative. Finally, we present a one-dimensional counterexample to illustrate the necessity of introducing a gap, smoothing, or regularity condition.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC