Auto-PyTorch

Multi-Fidelity MetaLearning for Efficient and Robust AutoDL

verfasst von

Lucas Zimmer, Marius Lindauer, Frank Hutter

Abstract

While early AutoML frameworks focused on optimizing traditional ML pipelines and their hyperparameters, a recent trend in AutoML is to focus on neural architecture search. In this paper, we introduce Auto-PyTorch, which brings the best of these two worlds together by jointly and robustly optimizing the architecture of networks and the training hyperparameters to enable fully automated deep learning (AutoDL). Auto-PyTorch achieves state-of-the-art performance on several tabular benchmarks by combining multi-fidelity optimization with portfolio construction for warmstarting and ensembling of deep neural networks (DNNs) and common baselines for tabular data. To thoroughly study our assumptions on how to design such an AutoDL system, we additionally introduce a new benchmark on learning curves for DNNs, dubbed LCBench, and run extensive ablation studies of the full Auto-PyTorch on typical AutoML benchmarks, eventually showing that Auto-PyTorch performs better than several state-of-the-art competitors on average.

Organisationseinheit(en)

Fachgebiet Maschinelles Lernen
Institut für Informationsverarbeitung

Externe Organisation(en)

Albert-Ludwigs-Universität Freiburg
Bosch Center for Artificial Intelligence (BCAI)

Typ

Artikel

Journal

IEEE Transactions on Pattern Analysis and Machine Intelligence

Band

43

Seiten

3079-3090

Anzahl der Seiten

12

ISSN

0162-8828

Publikationsdatum

01.09.2021

Publikationsstatus

Veröffentlicht

Peer-reviewed

Ja

ASJC Scopus Sachgebiete

Software, Artificial intelligence, Angewandte Mathematik, Maschinelles Sehen und Mustererkennung, Theoretische Informatik und Mathematik

Elektronische Version(en)

https://arxiv.org/pdf/2006.13799 (Zugang: Offen)
https://doi.org/10.1109/TPAMI.2021.3067763 (Zugang: Geschlossen)