CMU Multilingual NLP - Machine Translation-Sequence-to-Sequence Models
Explore machine translation and sequence-to-sequence models in this 44-minute lecture from CMU's Multilingual Natural Language Processing course. Delve into conditional language modeling, simple sequence-to-sequence models, generation methods, attention mechanisms, and self-attention/transformers. Learn about calculating sentence probabilities, hidden state passing, and various generation techniques including ancestral sampling, greedy search, and beam search. Examine sentence representations, attention score functions, and the distinction between attention and alignment. Discover multi-headed attention, supervised training approaches, and the intricacies of self-attention in Transformer models. Gain insights into Transformer training tricks, masking techniques for efficient training, and a unified view of sequence-to-sequence models. Conclude with a code walkthrough to solidify understanding of these advanced NLP concepts.