Explore a thought-provoking lecture on the evolution of machine learning theory, from classical statistics to modern deep learning approaches. Delve into key concepts such as Empirical Risk Minimization, uniform laws of large numbers, and capacity control. Examine the intriguing U-shaped generalization curve and challenge conventional wisdom about overfitting in interpolation. Discover why deep learning interpolation has become best practice and investigate the "double descent" risk curve phenomenon. Analyze random Fourier networks, interpolated k-NN schemes, and the relationship between interpolation and adversarial examples. Gain insights into the landscape of generalization, optimization techniques, and the power of interpolation in modern machine learning frameworks. Learn valuable lessons from deep learning applications and explore the potential of fast and effective kernel machines.