Bayesian Optimization with a Prior for the Optimum

verfasst von

Artur Souza, Luigi Nardi, Leonardo B. Oliveira, Kunle Olukotun, Marius Lindauer, Frank Hutter

Abstract

While Bayesian Optimization (BO) is a very popular method for optimizing expensive black-box functions, it fails to leverage the experience of domain experts. This causes BO to waste function evaluations on bad design choices (e.g., machine learning hyperparameters) that the expert already knows to work poorly. To address this issue, we introduce Bayesian Optimization with a Prior for the Optimum (BOPrO). BOPrO allows users to inject their knowledge into the optimization process in the form of priors about which parts of the input space will yield the best performance, rather than BO’s standard priors over functions, which are much less intuitive for users. BOPrO then combines these priors with BO’s standard probabilistic model to form a pseudo-posterior used to select which points to evaluate next. We show that BOPrO is around 6.67 × faster than state-of-the-art methods on a common suite of benchmarks, and achieves a new state-of-the-art performance on a real-world hardware design application. We also show that BOPrO converges faster even if the priors for the optimum are not entirely accurate and that it robustly recovers from misleading priors.

Organisationseinheit(en)

Fachgebiet Maschinelles Lernen
Institut für Informationsverarbeitung

Externe Organisation(en)

Universidade Federal de Minas Gerais
Lund University
Stanford University
Albert-Ludwigs-Universität Freiburg
Bosch Center for Artificial Intelligence (BCAI)

Typ

Aufsatz in Konferenzband

Band

3

Seiten

265-296

Anzahl der Seiten

32

Publikationsdatum

2021

Publikationsstatus

Veröffentlicht

Peer-reviewed

Ja

ASJC Scopus Sachgebiete

Theoretische Informatik, Informatik (insg.)

Elektronische Version(en)

https://arxiv.org/abs/2006.14608 (Zugang: Offen)
https://doi.org/10.1007/978-3-030-86523-8_17 (Zugang: Geschlossen)