Studying Generalization in Deep Learning via PAC-Bayes
Explore the intersection of generalization theory and deep learning in this 45-minute lecture from the Frontiers of Deep Learning series. Delve into PAC-Bayes theory and its applications to risk bounds for Gibbs classifiers and deterministic classifiers. Examine distribution-dependent approximations of optimal priors, the role of privacy, and the use of SGD to predict SGD. Investigate data and distribution priors for neural networks, focusing on MNIST results with coupled data-dependent priors and posteriors. Analyze bounds with oracle covariance and ghost samples, comparing results across different sample sizes. Gain insights into the potential contributions of generalization theory to deep learning and the challenges in explaining generalization.