Interpolation does not overfit even for very noisy data
7
Deep learning practice
8
Generalization theory for interpolation?
9
A way forward?
10
Interpolated k-NN schemes
11
Interpolation and adversarial examples
12
"Double descent" risk curve
13
what is the mechanism?
14
Double Descent in Linear regression
15
Occams's razor
16
The landscape of generalization
17
where is the interpolation threshold?
18
Optimization under interpolation
19
SGD under interpolation
20
The power of interpolation
21
Learning from deep learning: fast and effective kernel machines
22
Important points
23
From classical statistics to modern ML
Description:
Explore the evolution from classical statistics to modern machine learning in this 50-minute lecture by Mikhail Belkin from The Ohio State University. Delve into supervised ML, generalization bounds, and the classical U-shaped generalization curve. Examine the concept of interpolation in deep learning, addressing whether it leads to overfitting and its effectiveness even with noisy data. Investigate the "double descent" risk curve, its mechanisms, and implications for linear regression. Analyze the landscape of generalization, optimization under interpolation, and the power of interpolation in modern ML techniques. Gain insights into fast and effective kernel machines inspired by deep learning, and understand the key points in the transition from classical statistical approaches to contemporary machine learning methodologies.
From Classical Statistics to Modern Machine Learning