Title: Demonstrating Lower Sample Requirements in Prior-Informed Hyperparameter Optimization

Abstract:

The automated machine learning (AutoML) field faces mounting scrutiny regarding scalability and energy consumption, largely due to the immense computational costs associated with large-scale hyperparameter optimization (HPO). While current approaches leverage prior information heuristically to speed up processes in both black-box and multi-fidelity contexts, there has been a lack of quantitative characterization regarding how the informativeness of these priors actually reduces sample complexity. This study introduces the first distribution-dependent sample complexity bounds for multi-fidelity HPO involving priors, analyzed through the formal framework of fixed-budget best-arm identification.

By treating prior distributions as direct models of arm means—representing configuration performance—we establish explicit, distribution-dependent error bounds. These bounds clarify the link between the evaluation budget and the utility of priors. Our theoretical analysis reveals that informative priors, which assign higher probability mass to arms near the optimum, significantly decrease the number of evaluations needed. Conversely, if priors are uninformative or misleading, performance reverts to baseline levels. We validated these theoretical findings through proof-of-concept experiments using a synthetic benchmark and LCBench, a standard multi-fidelity HPO benchmark for deep learning. The results demonstrate a potential budget reduction of up to 90% without compromising solution quality. Collectively, this work establishes a rigorous, principled basis for developing compute-efficient, green AutoML systems driven by prior guidance.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC