Главная
Study mode:
on
1
Intro
2
Generalization via Bis-Variance Tradeoff
3
Generalization in Deep Learning
4
Linear Models: Implicit Norm Minimization Linear Regression
5
Implicit Norm Minimization In Deep Learning?
6
Perspective: Implicit Rank Minimization
7
Outline
8
Matrix Completion Two-Dimensional Prediction
9
MF Linear NN
10
Conjecture: Implicit Nuclear Norm Minimization
11
Dynamical Analysis of Implicit Regularization in MF
12
Implicit Regularization in MF Norm Minimization Does the implicit regularization in MF minimize a norm?
13
Drawbacks of Studying MF
14
Tensor Completion Multi-Dimensional Prediction
15
TF Shallow Non-Linear Convolutional NN
16
Dynamical Analysis of Implicit Regularization in TF
17
Analogy Between Implicit Regularizations
18
HTF Deep Non-Linear CNN TF does not account for depth
19
Dynamical Analysis of Implicit Regularization in HTF
20
Practical Application: Rank Minimization in NN Layers
21
Potential Explanation for Generalization on Natural Data
22
Countering Locality of CNNs via Regularization
23
Recap
24
Implicit Rank Minimization in Deep Learning
Description:
Explore a lecture on the implicit regularization in deep learning through the lens of rank minimization. Delve into the theoretical analysis of matrix and tensor factorizations, equivalent to certain linear and non-linear neural networks. Discover how gradient-based optimization tends to fit training data with predictors of low complexity, leading to generalization. Examine dynamical characterizations establishing implicit regularization towards low matrix and tensor ranks, challenging prior beliefs about norm minimization. Consider the implications of these findings for both theory and practice in modern deep learning, highlighting the potential of ranks to explain and improve generalization. Learn about matrix completion, tensor completion, and their connections to linear and non-linear neural networks. Investigate practical applications, including rank minimization in neural network layers and potential explanations for generalization on natural data.

Generalization in Deep Learning Through the Lens of Implicit Rank Minimization

Hausdorff Center for Mathematics
Add to list