PAC-Bayesian Approaches to Understanding Generalization in Deep Learning - Gintare Dziugaite
Explore PAC-Bayesian approaches to understanding generalization in deep learning through this 32-minute lecture from the Workshop on Theory of Deep Learning: Where next? Delve into topics such as PAC-Bayes generalization bounds, distribution-dependent priors, data-dependent oracle priors for neural networks, and Gaussian network bounds. Learn about the setup, outline, and various PAC-Bayes concepts, including risk bounds for Gibbs classifiers and deterministic classifiers. Examine the empirical evaluation of Lever et al.'s bounds and investigate distribution-dependent approximations of optimal priors via privacy. Gain insights into directly optimizing Variational data-dependent PAC-Bayes generalization bounds. Presented by Gintare Karolina Dziugaite from the Simons Institute for the Theory of Computing, this talk offers a comprehensive overview of PAC-Bayesian approaches in deep learning generalization.