DACBench: A Benchmark Library for Dynamic Algorithm Configuration

verfasst von

Theresa Eimer, André Biedenkapp, Maximilian Reimer, Steven Adriaensen, Frank Hutter, Marius Thomas Lindauer

Abstract

Dynamic Algorithm Configuration (DAC) aims to dynamically control a target algorithm's hyperparameters in order to improve its performance. Several theoretical and empirical results have demonstrated the benefits of dynamically controlling hyperparameters in domains like evolutionary computation, AI Planning or deep learning. Replicating these results, as well as studying new methods for DAC, however, is difficult since existing benchmarks are often specialized and incompatible with the same interfaces. To facilitate benchmarking and thus research on DAC, we propose DACBench, a benchmark library that seeks to collect and standardize existing DAC benchmarks from different AI domains, as well as provide a template for new ones. For the design of DACBench, we focused on important desiderata, such as (i) flexibility, (ii) reproducibility, (iii) extensibility and (iv) automatic documentation and visualization. To show the potential, broad applicability and challenges of DAC, we explore how a set of six initial benchmarks compare in several dimensions of difficulty.

Organisationseinheit(en)

Fachgebiet Automatische Bildinterpretation
Fachgebiet Maschinelles Lernen
Institut für Informationsverarbeitung

Externe Organisation(en)

Albert-Ludwigs-Universität Freiburg

Typ

Aufsatz in Konferenzband

Seiten

1668-1674

Anzahl der Seiten

7

Publikationsdatum

2021

Publikationsstatus

Veröffentlicht

Peer-reviewed

Ja

ASJC Scopus Sachgebiete

Artificial intelligence

Elektronische Version(en)

https://arxiv.org/abs/2105.08541 (Zugang: Offen)
https://doi.org/10.24963/ijcai.2021/230 (Zugang: Offen)