Parameter Sharing - Recurrent and Convolutional Nets
Explore parameter sharing in recurrent and convolutional neural networks in this comprehensive 2-hour lecture by Yann LeCun. Delve into hypernetworks, shared weights, and gradient addition in parameter sharing. Examine recurrent nets, including unrolling in time, vanishing and exploding gradients, and RNN tricks. Investigate memory concepts, LSTM networks, and attention mechanisms for sequence-to-sequence mapping. Study convolutional nets, including motif detection, convolution definitions, backpropagation, and architecture. Learn about vintage ConvNets, brain image interpretation, and the Hubel & Wiesel model of the visual cortex. Gain insights into ConvNet invariance and equivariance, training time, iteration cycles, and historical remarks in deep learning.