Dynamic Algorithm Configuration: Foundation of a New Meta-Algorithmic Framework

authored by

André Biedenkapp, H. Furkan Bozkurt, Theresa Eimer, Frank Hutter, Marius Thomas Lindauer

Abstract

The performance of many algorithms in the fields of hard combinatorial problem solving, machine learning or AI in general depends on parameter tuning. Automated methods have been proposed to alleviate users from the tedious and error-prone task of manually searching for performance-optimized configurations across a set of problem instances. However, there is still a lot of untapped potential through adjusting an algorithm's parameters online since different parameter values can be optimal at different stages of the algorithm. Prior work showed that reinforcement learning is an effective approach to learn policies for online adjustments of algorithm parameters in a data-driven way. We extend that approach by formulating the resulting dynamic algorithm configuration as a contextual MDP, such that RL not only learns a policy for a single instance, but across a set of instances. To lay the foundation for studying dynamic algorithm configuration with RL in a controlled setting, we propose white-box benchmarks covering major aspects that make dynamic algorithm configuration a hard problem in practice and study the performance of various types of configuration strategies for them. On these white-box benchmarks, we show that (i) RL is a robust candidate for learning configuration policies, outperforming standard parameter optimization approaches, such as classical algorithm configuration; (ii) based on function approximation, RL agents can learn to generalize to new types of instances; and (iii) self-paced learning can substantially improve the performance by selecting a useful sequence of training instances automatically.

Organisation(s)

Institute of Information Processing
Machine Learning Section

External Organisation(s)

University of Freiburg
Robert Bosch GmbH

Type

Conference contribution

Pages

427-434

No. of pages

8

Publication date

24.08.2020

Publication status

Published

Peer reviewed

Yes

ASJC Scopus subject areas

Artificial Intelligence

Electronic version(s)

https://doi.org/10.3233/FAIA200122 (Access: Open)