Title: Local Preferential Bayesian Optimization

Abstract: While Bayesian optimization (BO) is widely recognized for its efficacy in tuning costly, noisy experiments, it typically necessitates the definition of an explicit objective function. Preferential BO (PBO) circumvents this need by deriving insights from pairwise human feedback; however, current methodologies often falter in efficiently optimizing problems beyond low and medium dimensions, largely due to their reliance on global search strategies. To overcome this constraint, we have engineered a suite of local PBO techniques that adapt core principles from high-dimensional BO to the preferential context. Specifically, these methods integrate trust-region and derivative-informed local search mechanisms with pairwise preference data, leveraging the first- and second-order derivatives of the Laplace-approximated Gaussian Process (GP) posterior. Our evaluation, which encompasses GP sample paths, standard optimization benchmarks, and policy-search scenarios, demonstrates that local PBO approaches are particularly potent in navigating high-dimensional spaces and complex terrains characterized by steep optima. When measured against global preference-based baselines, our proposed methods significantly lower cumulative regret, highlighting their potential value for practical preference-based optimization applications, including policy search.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC