Prior-guided Bayesian Optimization

verfasst von

Artur Souza, Luigi Nardi, Leonardo B. Oliveira, Kunle Olukotun, Marius Lindauer, Frank Hutter

Abstract

While Bayesian Optimization (BO) is a very popular method for optimizing expensive black-box functions, it fails to leverage the experience of domain experts. This causes BO to waste function evaluations on commonly known bad regions of design choices, e.g., hyperparameters of a machine learning algorithm. To address this issue, we introduce Prior-guided Bayesian Optimization (PrBO). PrBO allows users to inject their knowledge into the optimization process in the form of priors about which parts of the input space will yield the best performance, rather than BO's standard priors over functions which are much less intuitive for users. PrBO then combines these priors with BO's standard probabilistic model to yield a posterior. We show that PrBO is more sample efficient than state-of-the-art methods without user priors and 10,000\(\times\) faster than random search, on a common suite of benchmarks and a real-world hardware design application. We also show that PrBO converges faster even if the user priors are not entirely accurate and that it robustly recovers from misleading priors.

Organisationseinheit(en)

Fachgebiet Maschinelles Lernen
Institut für Informationsverarbeitung

Externe Organisation(en)

Universidade Federal de Minas Gerais
Lund University
Stanford University
Albert-Ludwigs-Universität Freiburg
Robert Bosch GmbH

Typ

Aufsatz in Konferenzband

Publikationsdatum

2021

Publikationsstatus

Elektronisch veröffentlicht (E-Pub)

Peer-reviewed

Ja

Elektronische Version(en)

https://arxiv.org/pdf/2006.14608 (Zugang: Offen)