Title: Riemannian Stochastic Optimization for Sufficient Dimension Reduction

Abstract:

Sufficient dimension reduction (SDR) facilitates high-dimensional regression by mapping covariates to a lower-dimensional subspace that retains the conditional mean of the response. Current gradient-based estimators face significant limitations: those functioning in the ambient space are hindered by the curse of dimensionality, while those localizing in the reduced space incur a per-outer-iteration cost that is at least quadratic relative to the sample size. This study demonstrates that minimizers of the population Minimum Average Variance Estimation (MAVE) risk approximate the same Grassmannian target as the Outer Product of Gradients (OPG) method. We reformulate the empirical criterion as a smooth maximization problem on the Stiefel manifold, featuring a closed-form Riemannian gradient. The proposed algorithm, SMAVE, integrates sparse projected-space nearest-neighbor localization with Riemannian stochastic gradient ascent. A simplified variant of this approach guarantees almost-sure convergence and achieves a non-asymptotic rate consistent with standard non-convex stochastic first-order scaling. Empirical evaluations show that SMAVE matches or surpasses RMAVE in synthetic subspace recovery for moderate-to-high ambient dimensions. Furthermore, across four real-world datasets, SMAVE consistently outperforms OPG and competes with or exceeds RMAVE’s performance while operating at runtimes that are orders of magnitude lower.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC