Главная
Study mode:
on
1
Awesome song and introduction
2
Loading the modules
3
Creating the training dataset
4
Coding Position Encoding
5
Coding Attention
6
Coding a Decoder-Only Transformer
7
Running the model untrained
8
Training and using the model
Description:
Walk through the process of coding a ChatGPT-like Transformer from scratch using PyTorch in this comprehensive 31-minute video tutorial. Learn how to load necessary modules, create a training dataset, implement position encoding, code attention mechanisms, and build a decoder-only Transformer. Observe the model running untrained before diving into the training process and practical application. Gain insights into the step-by-step implementation with clear explanations of every detail, assuming prior knowledge of decoder-only Transformers, essential matrix algebra for neural networks, and matrix math behind Transformers.

Coding a ChatGPT-Like Transformer From Scratch in PyTorch

StatQuest with Josh Starmer
Add to list
0:00 / 0:00