Главная
Study mode:
on
1
Intro
2
Remember: Neural Models
3
How to Train Embeddings?
4
What do we want to know about words?
5
Contextualization of Word Representations
6
A Manual Attempt: WordNet
7
An Answer (?): Word Embeddings!
8
Word Embeddings are Cool! (An Obligatory Slide)
9
Distributional vs. Distributed Representations
10
Distributional Representations (see Goldberg 10.4.1)
11
Count-based Methods
12
Prediction-basd Methods (See Goldberg 10.4.2)
13
Word Embeddings from Language Models giving
14
Context Window Methods
15
Glove (Pennington et al. 2014)
16
What Contexts?
17
Types of Evaluation
18
Non-linear Projection • Non-linear projections group things that are close in high
19
t-SNE Visualization can be Misleading! Wattenberg et al. 2016
20
Intrinsic Evaluation of Embeddings (categorization from Schnabel et al 2015)
21
Extrinsic Evaluation
22
How Do I Choose Embeddings?
23
When are Pre-trained Embeddings Useful?
24
Limitations of Embeddings
25
Unsupervised Coordination of Embeddings
26
Retrofitting of Embeddings to Existing Lexicons . We have an existing lexicon like WordNet, and would like our vectors to match (Faruqui et al. 2015)
27
Sparse Embeddings
28
De-biasing Word
29
FastText Toolkit
Description:
Learn about distributional semantics and word vectors in this comprehensive lecture from CMU's Neural Networks for NLP course. Explore techniques for describing words by their context, including counting and prediction methods. Dive into skip-grams, continuous bag-of-words (CBOW), and advanced word vector approaches. Discover methods for evaluating and visualizing word vectors, and gain insights into their limitations and applications. Examine topics such as contextualization, WordNet, GloVe, intrinsic and extrinsic evaluation, pre-trained embeddings, and techniques for improving embeddings like retrofitting and de-biasing.

Neural Nets for NLP 2021 - Distributional Semantics and Word Vectors

Graham Neubig
Add to list
#Computer Science #Artificial Intelligence #Neural Networks #Natural Language Processing (NLP) #Machine Learning #Word Embeddings #Language Models #Word Vectors
1 of 29

Intro