Title: Achieving Regret Guarantees in Bayesian Optimization via Optimal-Point Variance Reduction

Abstract:

This study investigates a one-step lookahead Bayesian optimization (BO) framework, focusing on its theoretical underpinnings. While the practical performance of one-step lookahead approaches, including entropy search, has been widely documented, these methods typically depend on computationally prohibitive approximations. Moreover, their theoretical regret bounds have not been thoroughly established. To address these limitations, we introduce a novel one-step lookahead BO algorithm named Optimal-Point Variance Reduction (OVR). This method is computationally efficient, relying solely on posterior sampling and Monte Carlo approximations. We derive a uniform error bound for the Monte Carlo estimation process within the OVR framework across the entire input domain. Additionally, we prove that a regularized variant of OVR, which incorporates minor adjustments to encourage exploration, attains a vanishing upper bound on the Bayesian expected simple regret. The efficacy of the proposed OVR method is further validated through a series of numerical experiments.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC