Neural Network Scaling Limits

Neural Network Scaling Limits

Explore the analytical study of neural networks through scaling limits in this comprehensive lecture by Boris Hanin from Princeton University. Delve into the simplified models of learning that emerge when structural network parameters like depth, width, and training data size approach infinity. Survey various approaches to understanding these scaling limits, gaining insight into the complex and not fully understood behaviors that arise when some or all network parameters are large. Recorded at IPAM's Theory and Practice of Deep Learning Workshop at UCLA, this hour-long presentation offers a deep dive into the theoretical foundations of neural network behavior at scale.