Publication Details

Learning Step-Size Adaptation in CMA-ES

authored by
Gresa Shala, André Biedenkapp, Noor Awad, Steven Adriaensen, Marius Lindauer, Frank Hutter

An algorithm’s parameter setting often affects its ability to solve a given problem, e.g., population-size, mutation-rate or crossover-rate of an evolutionary algorithm. Furthermore, some parameters have to be adjusted dynamically, such as lowering the mutation-strength over time. While hand-crafted heuristics offer a way to fine-tune and dynamically configure these parameters, their design is tedious, time-consuming and typically involves analyzing the algorithm’s behavior on simple problems that may not be representative for those that arise in practice. In this paper, we show that formulating dynamic algorithm configuration as a reinforcement learning problem allows us to automatically learn policies that can dynamically configure the mutation step-size parameter of Covariance Matrix Adaptation Evolution Strategy (CMA-ES). We evaluate our approach on a wide range of black-box optimization problems, and show that (i) learning step-size policies has the potential to improve the performance of CMA-ES; (ii) learned step-size policies can outperform the default Cumulative Step-Size Adaptation of CMA-ES; and transferring the policies to (iii) different function classes and to (iv) higher dimensions is also possible.

Machine Learning Section
Institute of Information Processing
External Organisation(s)
University of Freiburg
Bosch Center for Artificial Intelligence (BCAI)
Conference contribution
No. of pages
Publication date
Publication status
Peer reviewed
ASJC Scopus subject areas
Theoretical Computer Science, Computer Science(all)
Electronic version(s) (Access: Closed)