It is well known that the choice of a best-possible solver for a given optimization problem crucially depends on the problem characteristics as well as on the available computational resources.
Understanding this dependency is a core research task in optimization. To support researchers and practitioners in choosing the right optimization approaches for their problem at hand, automated machine learning (AutoML) proposes a plethora of algorithm selection and configuration techniques that are explicitly trained to recommend the most suitable algorithms for a given optimization task. Until very recently, however, these AutoML approaches were restricted to a static selection of algorithm and configuration, thereby ignoring the fact that the best choices can drastically change during the optimization process, e.g., by requiring exploration in the early phases while preferring exploitation as more information about the problem at hand becomes available. Dynamic Algorithm Configuration (DAC) addresses this shortcoming by explicitly asking for a state-dependent selection of algorithms and configurations.
The Opt4DAC project will bring together experts in Optimization and in AutoML to leverage the untouched potential of DAC. We aim specifically to come up with approaches that identify suitable switches between different solvers and their configurations on the fly, i.e., in direct response to the requirements of the current optimization progress. To this end, we aim for (i) warm-starting optimizer with information obtained from other solvers, (ii) designing relevant features to describe the current state of the optimization process, and (iii) mechanisms to identify the best moment to reconfigure solvers. Theoretical insights will be challenged through extensive empirical evaluations, ranging from configuring evolutionary algorithms for solving traditional black-box optimization problems to Bayesian Optimization for Hyperparameter Optimization of Deep Neural Networks.
Lead at LUHAI: Prof. Lindauer
Lead at Sorbonne Université: Carola Doerr (CNRS/LIP6 Research Director)
Funding Program: DFG-ANR
Project Period: January 2024 to December 2026
