Главная
Study mode:
on
1
uTransfer introduced
2
Previous work tensor programs IV
3
NTK - neural tangent kernel recap
4
abc parametrization
5
How does learning happen in NTK?
6
Connections to Central Limit Theorem
7
Maximal Update Parametrization in Practice
8
DeepNet paper connection
9
Results width is all you need?
Description:
Dive into an in-depth exploration of the "Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer (μTransfer)" paper in this comprehensive video lecture. Learn about the groundbreaking approach that makes optimal hyperparameters stable with respect to width scaling. Explore previous work on tensor programs, revisit the neural tangent kernel (NTK) concept, and understand the abc parametrization. Discover how learning occurs in NTK and its connections to the Central Limit Theorem. Gain practical insights into Maximal Update Parametrization and its relationship to the DeepNet paper. Analyze the results that suggest width might be the key factor in neural network performance. Enhance your understanding of advanced machine learning concepts and cutting-edge research in neural network optimization.

Tensor Programs - Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer

Aleksa Gordić - The AI Epiphany
Add to list
0:00 / 0:00