Publication Details


HyperBand + Human Expert Knowledge

authored by
Neeratyoy Mallik, Carl Hvarfner, Danny Stoll, Maciej Janowski, Eddie Bergman, Marius Thomas Lindauer, Luigi Nardi, Frank Hutter

Hyperparameters of Deep Learning (DL) pipelines are crucial for their performance. While a large number of methods for hyperparameter optimization (HPO) have been developed, they are misaligned with the desiderata of a modern DL researcher. Since often only a few trials are possible in the development of new DL methods, manual experimentation is still the most prevalent approach to set hyperparameters, relying on the researcher’s intuition and cheap preliminary explorations. To resolve this shortcoming of HPO for DL, we propose PriorBand, an HPO algorithm tailored to DL, able to utilize both expert beliefs and cheap proxy tasks. Empirically, we demonstrate the efficiency of PriorBand across a range of DL models and tasks using as little as the cost of 10 training runs and show its robustness against poor expert beliefs and misleading proxy tasks

Institute of Artificial Intelligence
External Organisation(s)
University of Freiburg
Lund University
Stanford University
Conference contribution
Publication date
Publication status
Peer reviewed
Electronic version(s) (Access: Open)