Title: Generalization Bounds and Distribution Regression in Semi-Supervised Learning with Noisy Proxy Covariates

Abstract: In contemporary machine learning workflows, it is common to possess a wealth of pretrained representations that function as noisy proxy covariates, even as task-specific labels are limited. This paper investigates semi-supervised regression within this specific context. We introduce a straightforward two-stage estimation method that extracts kernel eigenfeatures from the entire set of proxy covates and subsequently applies a ridge predictor to the labeled data. Our theoretical analysis yields finite sample bounds, demonstrating that the method achieves fast convergence rates with respect to labeled samples, provided that perturbations in the proxy data are managed and a sufficient volume of unlabeled proxy covariates is available. Furthermore, we establish that distribution regression emerges as a direct special case of this framework, offering comparable performance guarantees when the finite bag size is adequately large. Empirical results confirm that our approach consistently outperforms both supervised and semi-supervised baseline methods, particularly in scenarios characterized by a scarcity of labeled data.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC