Title: The Global Convergence of Adaptive Sensing in Principal Eigenvector Estimation

Original: arXiv:2505.10882v2 Announce Type: replace Abstract: Principal component analysis classically requires full $d$-dimensional samples, yet in various applications hardware limits acquisition to a few scalar measurements per sample. We analyze a compressed variant of Oja's algorithm for estimating the principal eigenvector of the data covariance matrix using only two adaptive measurements per sample. At each iteration, we observe one measurement along the current estimate and one in a random orthogonal direction. We prove that after $t$ iterations, the expected sine-squared error to the true eigenvector is $\mathcal{O}(\lambda_1\lambda_2 d^2 / (\Delta^2 t))$, where $d$ is the ambient dimension, $\lambda_1, \lambda_2$ are the leading eigenvalues, and $\Delta = \lambda_1 - \lambda_2$ is the eigengap. We complement this with a matching information-theoretic lower bound of $\Omega(\lambda_1\lambda_2 d^2 / (\Delta^2 t))$ -- the first for compressed eigenvector estimation -- proving that the $d^2$ factor, an additional factor of $d$ compared to the fully-observed minimax rate $\Theta(\lambda_1\lambda_2 d / (\Delta^2 t))$, is the fundamental cost of compression and cannot be improved. In contrast, any non-adaptive scheme with two measurements per iteration suffers $\Omega(\lambda_2^2 d^3 / (\Delta^2 t))$, an additional power of $d$. This separates fully-observed, adaptive-compressed, and non-adaptive-compressed PCA across three powers of $d$. Our analysis handles the noisy setting where the covariance has nonzero trailing eigenvalues, providing the first convergence guarantee for adaptive compressed subspace tracking beyond the noiseless case.

Rewritten: Title: Global Convergence of Adaptive Sensing for Principal Eigenvector Estimation

Abstract: While traditional principal component analysis demands complete $d$-dimensional observations, practical hardware constraints in many scenarios restrict data acquisition to merely a few scalar values per sample. This study examines a compressed version of Oja’s algorithm designed to estimate the principal eigenvector of the data covariance matrix, utilizing just two adaptive measurements for each sample. Specifically, each iteration involves recording one measurement aligned with the present estimate and another in a direction randomly chosen to be orthogonal. We demonstrate that following $t$ iterations, the anticipated sine-squared deviation from the actual eigenvector scales as $\mathcal{O}(\lambda_1\lambda_2 d^2 / (\Delta^2 t))$. Here, $d$ denotes the ambient dimension, $\lambda_1$ and $\lambda_2$ represent the top two eigenvalues, and $\Delta = \lambda_1 - \lambda_2$ signifies the eigengap. To support this finding, we establish a corresponding information-theoretic lower bound of $\Omega(\lambda_1\lambda_2 d^2 / (\Delta^2 t))$, marking the first such bound for compressed eigenvector estimation. This confirms that the $d^2$ term represents an inherent penalty of compression—adding a factor of $d$ to the fully observed minimax rate of $\Theta(\lambda_1\lambda_2 d / (\Delta^2 t))$—and is irreducible. Conversely, non-adaptive approaches employing two measurements per iteration incur an error of at least $\Omega(\lambda_2^2 d^3 / (\Delta^2 t))$, introducing an extra power of $d$. These results distinguish fully observed, adaptively compressed, and non-adaptively compressed PCA across three distinct powers of $d$. Furthermore, our framework addresses noisy environments characterized by nonzero trailing eigenvalues in the covariance, delivering the inaugural convergence guarantee for adaptive compressed subspace tracking in scenarios beyond the noiseless assumption.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC