Beyond Lazy Training for Over-parameterized Tensor Decomposition

Beyond Lazy Training for Over-parameterized Tensor Decomposition

Explore a lecture on over-parameterized tensor decomposition and its applications beyond lazy training. Delve into the mathematical foundations and algorithms for tensor computations, focusing on how gradient descent variants can find approximate tensor decompositions. Learn about the limitations of lazy training regimes, the challenges in analyzing gradient descent, and a novel high-level algorithm that overcomes these obstacles. Discover how this research relates to training neural networks and utilizing low-rank structure in data. Gain insights into the proof ideas, including maintaining iterates close to the correct subspace and escaping local minima through random correlation and tensor power methods.