AutoML

Prof. Dr. rer. nat. Marius Lindauer

---

Übungsbetreuung:

Background

Machine Learning (ML) and in particular Deep Learning (DL) achieved many breakthroughs in the last years. Unfortunately, ML/DL does not only require data, but it requires also a lot of expert knowledge, if you want to apply it successfully---even experts in ML/DL still need a lot of time to do it. Major challenges for new ML/DL applications include the choice of the algorithm to be used (SVM, random forest, deep neural networks) and its hyper-parameter settings (e.g., kernel coefficient of a RBF-SVM). Unfortunately, to obtain accurate predictions, these design decisions are crucial and have to be made for each dataset. This is particularly hard for deep learning, where we have to choose a well-performing architecture of the network and for example to set the hyper-parameters of the optimizer (e.g., learning rate). Since training deep neural networks often requires quite some time (minutes, hours or even weeks), we cannot exhaustively try several networks architectures and hyper-parameter configurations, but we have to find more efficient approaches. Overall, all these design decisions require a lot of expert knowledge, the process takes quite some time and the manual tuning is a tedious and error-prone task. We will discuss approaches and meta-systems, that automate the process of obtaining well-performing machine learning systems, so-called Automated Machine Learning (AutoML). These AutoML systems allow for faster development of new ML/DL applications, require far less expert knowledge than doing everything from scratch and often even outperform human developers. In this lecture, you will learn how to use such AutoML systems, to develop your own systems and to understand ideas behind state-of-the-art AutoML approaches.

Requirements

We strongly recommend that you know the foundations of

• machine learning
• and deep learning

in order to attend the course. You should have attended at least one other course for ML and DL in the past.

Topics

The lectures are partioned in several parts, including:

• Hyperparameter optimization
• Neural architecture search
• Meta-learning
• Dynamic algorithm configuration

Literature

• Automated Machine Learning by Hutter, Kotthoff and Vanschoren (open access book)

Dynamics

The course will be in English.

The course will be taught in a flipped-classroom style. We will meet weekly once for a combined Q/A session and an exercise. Roughly every week, there will be a new exercise sheet. Most exercises will be practical and involve teamwork (teams of 2 students!) so that you learn how to apply AutoML in practice.

• Lecture/Exercise: Wednesday 14:00 (s.t.) - 15:30am, online-only!