Главная
Study mode:
on
1
Intro
2
Sentence Representations
3
Basic Idea (Bahdanau et al. 2015)
4
Calculating Attention (1)
5
A Graphical Example
6
Attention Score Functions (2)
7
Input Sentence
8
Previously Generated Things
9
Various Modalities
10
Hierarchical Structures (Yang et al. 2016)
11
Multiple Sources
12
Intra-Attention / Self Attention (Cheng et al. 2016) • Each element in the sentence attends to other elements + context sensitive encodings!
13
Coverage
14
Incorporating Markov Properties (Cohn et al. 2015)
15
Bidirectional Training (Cohn et al. 2015)
16
Supervised Training (Mi et al. 2016)
17
Attention is not Alignment! (Koehn and Knowles 2017) • Attention is often blurred
18
Monotonic Attention (e.g. Yu et al. 2016)
19
Convolutional Attention (Allamanis et al. 2016)
20
Multi-headed Attention
21
Summary of the "Transformer" (Vaswani et al. 2017)
22
Attention Tricks
Description:
Explore a comprehensive lecture on attention mechanisms in neural networks for natural language processing. Delve into the fundamentals of attention, including what to attend to, improvements to attention, and specialized attention varieties. Examine a case study on the "Attention is All You Need" paper. Learn about various attention score functions, hierarchical structures, multiple sources, and intra-attention. Discover advanced concepts such as coverage, bidirectional training, supervised training, and monotonic attention. Investigate convolutional attention and multi-headed attention, concluding with a summary of the "Transformer" model and useful attention tricks. Access accompanying slides and code examples to enhance your understanding of these crucial NLP concepts.

Neural Nets for NLP 2017 - Attention

Graham Neubig
Add to list
#Computer Science #Artificial Intelligence #Neural Networks #Natural Language Processing (NLP) #Deep Learning #Attention Mechanisms #Machine Learning #Transformer Models #Self-Attention
0:00 / 0:00
1 of 22

Intro