Title: Reassessing Privacy Amplification by Subsampling in Selective Release DPSGD

Original: arXiv:2606.04384v1 Announce Type: new Abstract: Machine learning's reliance on sensitive data necessitates privacy-preserving techniques like Differentially Private Stochastic Gradient Descent (DPSGD). However, DPSGD suffers from substantial utility degradation and slow convergence due to gradient clipping and noise injection. Prior works have attempted to improve DPSGD from various perspectives; notably, the Differentially Private Selective Update and Release (DPSUR) algorithm has achieved remarkable model utility. However, the privacy accounting in DPSUR overlooks the variation in sampling probability introduced by the selective release mechanism, which compromises the rigor of its privacy guarantees. To address these limitations, we re-evaluate the privacy analysis of the selective release mechanism and propose a novel algorithm: Differentially Private Selective Release based on Clipped Gradients (DPSR-CG). Through a rigorous, newly derived privacy analysis and extensive experiments on multiple datasets (MNIST, CIFAR-10, IMDB, and FMNIST), we demonstrate that our DPSR-CG mechanism maintains strict privacy guarantees while achieving exceptional model performance.

Rewrite: The dependence of machine learning on sensitive information has led to the adoption of privacy-preserving methods such as Differentially Private Stochastic Gradient Descent (DPSGD). Nevertheless, the practice of injecting noise and clipping gradients often results in significant drops in utility and sluggish convergence. While previous research has sought to enhance DPSGD through different approaches, the Differentially Private Selective Update and Release (DPSUR) algorithm stands out for delivering impressive model utility. Yet, the privacy accounting within DPSUR fails to account for the fluctuations in sampling probability caused by the selective release process, thereby undermining the strictness of its privacy assurances. In response to these shortcomings, this study revisits the privacy analysis of the selective release mechanism and introduces a new algorithm called Differentially Private Selective Release based on Clipped Gradients (DPSR-CG). By employing a newly developed, rigorous privacy analysis and conducting comprehensive tests across several datasets—including MNIST, CIFAR-10, IMDB, and FMNIST—we show that the DPSR-CG approach upholds robust privacy protections while delivering outstanding model performance.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC