A Function Space View of Overparameterized Neural Networks - Rebecca Willet, University of Chicago
Explore a function space perspective on overparameterized neural networks in this 35-minute conference talk by Rebecca Willet from the University of Chicago. Delve into the intriguing phenomenon of vastly overparameterized neural networks generalizing well despite their capacity to fit any labels. Examine the role of weight magnitude control in complexity regulation and investigate the functions approximated by neural networks with bounded weight norms. Discover a precise characterization of functions realizable by two-layer ReLU networks with bounded Euclidean norm weights, drawing surprising connections to the Radon transform used in computational imaging. Learn how Radon transform analysis provides novel insights into learning with two and three-layer ReLU networks. Gain understanding of topics such as infinite-width ReLU nets, norm-controlled learning, convex nets, and depth separation results. Conclude with open questions in this cutting-edge area of machine learning research presented at the Alan Turing Institute. Read more