Title: A Robust Optimization Approach to Sparse Principal Component Analysis

Abstract:

Although Principal Component Analysis (PCA) is a cornerstone technique for reducing data dimensionality, its reliance on dense representations renders it ineffective for high-dimensional datasets. Current solutions attempt to enforce sparsity via explicit $\ell_1$-penalties; however, tuning these penalties is challenging because the task is unsupervised. To overcome this, we introduce Adversarial PCA (AdvPCA), a method that employs robust optimization to induce sparsity. Specifically, AdvPCA optimizes the reconstruction objective by considering bounded, worst-case perturbations within the latent space. We demonstrate that this framework can be reduced to a closed-form solution, yielding a practical iterative algorithm. This procedure alternates between adversarial linear regression-style updates for the sparse encoder and orthogonal updates for the decoder. Furthermore, we provide a theoretical characterization of the solution, which leads to a data-adaptive parameterization, enabling the algorithm to perform well without extensive manual tuning. We substantiate these findings with numerical experiments conducted on both synthetic datasets and real-world genomics data.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC