Главная
Study mode:
on
1
Intro
2
The deep learning revolution recent examp
3
Deep Learning The story we all tell: deep learning algorithms build hierarchical models of input date, where the earlier layers create simple features and layer layers create high- level abstractions…
4
This talk
5
Outline
6
From deep networks to DEQs
7
Long history of related work
8
Implementing DEQS
9
The DEQ forward pass
10
How to train your DEQ
11
How to train your DED Compute gradients analytically via implicit function theorem
12
More information on implicit layers
13
Language modeling: WikiText-103
14
Multiscale deep equilibrium models
15
ImageNet Top-1 Accuracy
16
Citiscapes mlou
17
Visualization of Segmentation
18
Theoretical/algorithmic challenges for DE
19
Key result
20
Proof sketch for simpler case
21
Monotone operator equilibrium network
22
Initial study: CIFAR10
23
Additional points on monotone DEOS
24
Final thoughts
Description:
Explore deep equilibrium models (DEQs) and monotone equilibrium models (monDEQs) in this 46-minute lecture by Zico Kolter for the International Mathematical Union. Delve into the monDEQ framework, which guarantees fixed point uniqueness and enables efficient operator splitting methods. Learn how to bound Lipschitz constants of monDEQ models, produce generalization bounds for these "infinitely deep" networks, and characterize their robustness. Discover the connections between monDEQs and mean-field inference, and understand how these approaches can formulate Boltzmann-machine-like models with guaranteed convergence to globally optimal solutions. Examine practical applications in language modeling and image segmentation, and gain insights into the theoretical and algorithmic challenges of DEQs.

Deep Neural Networks via Monotone Operators

International Mathematical Union
Add to list