Learning Step-Size Adaptation in CMA-ES

authored by

Gresa Shala, André Biedenkapp, Noor Awad, Steven Adriaensen, Marius Lindauer, Frank Hutter

Abstract

An algorithm’s parameter setting often affects its ability to solve a given problem, e.g., population-size, mutation-rate or crossover-rate of an evolutionary algorithm. Furthermore, some parameters have to be adjusted dynamically, such as lowering the mutation-strength over time. While hand-crafted heuristics offer a way to fine-tune and dynamically configure these parameters, their design is tedious, time-consuming and typically involves analyzing the algorithm’s behavior on simple problems that may not be representative for those that arise in practice. In this paper, we show that formulating dynamic algorithm configuration as a reinforcement learning problem allows us to automatically learn policies that can dynamically configure the mutation step-size parameter of Covariance Matrix Adaptation Evolution Strategy (CMA-ES). We evaluate our approach on a wide range of black-box optimization problems, and show that (i) learning step-size policies has the potential to improve the performance of CMA-ES; (ii) learned step-size policies can outperform the default Cumulative Step-Size Adaptation of CMA-ES; and transferring the policies to (iii) different function classes and to (iv) higher dimensions is also possible.

Organisation(s)

Machine Learning Section
Institute of Information Processing

External Organisation(s)

University of Freiburg
Bosch Center for Artificial Intelligence (BCAI)

Type

Conference contribution

Pages

691-706

No. of pages

16

Publication date

2020

Publication status

Published

Peer reviewed

Yes

ASJC Scopus subject areas

Theoretical Computer Science, Computer Science(all)

Electronic version(s)

https://doi.org/10.1007/978-3-030-58112-1_48 (Access: Closed)