This module introduces the theoretical foundations of machine learning, with a focus on learning theory and its role in understanding the performance of learning algorithms. Students will learn how to formally analyze the generalization ability of models, understand the role of hypothesis spaces, and derive guarantees on learning performance. The course covers both classical learning theory and modern theoretical approaches that have emerged in response to the success of deep learning, where many classical assumptions break down. We will critically examine where traditional tools such as uniform convergence and VC-dimension fall short, and explore recent perspectives that aim to explain the generalization behavior of highly overparameterized models. Topics include, among others:
- Probably Approximately Correct (PAC)
- Hypothesis classes, loss functions, and empirical risk minimization
- Generalization bounds via VC-dimension, Rademacher complexity, and uniform convergence
- Algorithmic stability and regularization
- The failure of classical theory in deep learning: memorization and double descent
- Modern approaches to deep learning theory
- Lehrende:r: Michael Kamp