Mad Max - Affine Spline Insights into Deep Learning
Explore the connections between deep neural networks and spline theory in this 48-minute lecture by Richard Baraniuk from Rice University. Delve into the fundamentals of deep nets and splines, focusing on max-affine splines (MAS) and max-affine spline operators (MASO). Examine spline approximation techniques and various types of splines. Investigate the MASO spline partition and its role in learning, as well as the geometry of MASO partitions. Discover how convolutional neural networks (CNNs) relate to local affine mappings and how deep nets function as matched filterbanks. Analyze concepts such as data memorization, deep net complexity, and the impact of data augmentation. Gain insights into piecewise affine nets and explore potential future directions in deep learning research.