Title: Stochastic Rounding Elevates Minor Singular Values

Over the last six years, stochastic rounding (SR) has reemerged as a prominent quantization technique for low-precision floating-point calculations, finding utility across fields ranging from numerical analysis to contemporary machine learning architectures. Previous studies indicated that SR functions as an implicit regularizer, specifically by boosting the minimum singular value of matrices with extreme aspect ratios, such as those that are very tall and thin or short and fat. This study significantly refines and broadens that perspective in two key ways. Firstly, we establish that SR’s regularization influence is not confined to matrices with extreme aspect ratios; it remains effective for matrices with constant aspect ratios as well. Secondly, we prove that SR does not just enhance the smallest singular value but rather raises entire groups of singular values located at the tail end of the spectrum. Collectively, these findings offer a more comprehensive view of stochastic rounding as a spectral regularizer, demonstrating that its impact reaches beyond extreme aspect ratios and influences a wider segment of the singular value distribution.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC