Neural Nets for NLP 2017 - Attention
Explore a comprehensive lecture on attention mechanisms in neural networks for natural language processing. Delve into the fundamentals of attention, including what to attend to, improvements to attention, and specialized attention varieties. Examine a case study on the "Attention is All You Need" paper. Learn about various attention score functions, hierarchical structures, multiple sources, and intra-attention. Discover advanced concepts such as coverage, bidirectional training, supervised training, and monotonic attention. Investigate convolutional attention and multi-headed attention, concluding with a summary of the "Transformer" model and useful attention tricks. Access accompanying slides and code examples to enhance your understanding of these crucial NLP concepts.