Title: Establishing a Provable Scaling Law in Meta-Learning Through Complexity Minimization

Abstract: In contemporary machine learning, pre-training has emerged as a cornerstone paradigm. A primary empirical advantage of this approach is the reduction in downstream sample complexity, which occurs as the volume of pre-training data expands. Nevertheless, current theoretical models fail to provide a comprehensive explanation for this observed phenomenon. To address this gap, we propose complexity minimization, a new meta-representation learning framework that facilitates the theoretical examination of this scaling dynamic. This method acquires representations by assessing the downstream model complexity optimal for each specific domain and subsequently minimizing the maximum such complexity across all source domains. Our comprehensive theoretical analysis, which covers the entire pipeline from pre-training to downstream regression, demonstrates that this framework provably reproduces the scaling behavior. Specifically, we prove that the error rate associated with few-shot adaptation decreases as the quantity of meta-training data increases. Furthermore, our empirical results indicate that integrating complexity regularization into established meta-learning techniques consistently enhances downstream sample efficiency.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC