Building Makemore - Building a WaveNet
Dive into an in-depth tutorial on building a WaveNet-like convolutional neural network architecture, expanding upon a 2-layer MLP from previous lessons. Explore the process of deepening the network with a tree-like structure, mirroring the WaveNet (2016) architecture from DeepMind. Gain valuable insights into torch.nn, its inner workings, and typical deep learning development processes. Follow along as the instructor navigates through documentation, manages multidimensional tensor shapes, and transitions between Jupyter notebooks and repository code. Learn how to implement and train the WaveNet model, address common bugs, and scale up the architecture. Discover the concept of dilated causal convolutions and their efficient implementation in deep learning models. Conclude with an experimental harness, discussions on improving model performance, and future directions for enhancing WaveNet on the given dataset.