Towards Better Understanding of Contrastive Learning
Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only! Grab it Explore a comprehensive presentation on contrastive learning delivered by Yuandong Tian from Meta for the Data Learning working group. Delve into the empirical success of deep models, self-supervised learning, and the formulation of contrastive learning. Examine the understanding of contrastive loss, the InfoNCE example, and coordinate-wise optimization. Discover a surprising connection to kernels and gain insights into nonlinear analysis. Investigate training dynamics, including 1-layer 1-node and multiple node nonlinear networks. Learn about conditional independence, global modulation, and feature emergence. Compare quadratic loss versus InfoNCE through experimental settings, model architecture, and evaluation metrics. This 58-minute talk, recorded on March 7, 2023, offers valuable insights for researchers and students developing new technologies based on Data Assimilation and Machine Learning.