Главная
Study mode:
on
1
Intro
2
Discriminative vs. Generative Models • Discriminative model: calculate the probability of output given
3
Quiz: What Types of Variables?
4
Why Latent Random Variables?
5
An Example (Goersch 2016)
6
Problem: Straightforward Sampling is Inefficient
7
Solution: "Inference Model" • Predict which latent point produced the data point using inference
8
Disconnect Between Samples and Objective
9
VAE Objective • We can create an optimizable objective matching our problem, starting with KL divergence
10
Interpreting the VAE Objective
11
Problem! Sampling Breaks Backprop
12
Solution: Re-parameterization Trick
13
Generating from Language Models
14
Motivation for Latent Variables . Allows for a consistent latent space of sentences?
15
Difficulties in Training
16
KL Divergence Annealing
17
Weaken the Decoder
18
Discrete Latent Variables?
19
Method 1: Enumeration
20
Reparameterization (Maddison et al. 2017, Jang et al. 2017)
21
Gumbel-Softmax • A way to soften the decision and allow for continuous gradients
22
Variational Models of Language Processing (Miao et al. 2016)
23
Controllable Text Generation (Hu et al. 2017)
24
Symbol Sequence Latent Variables (Miao and Blunsom 2016)
Description:
Explore models with latent random variables in neural networks for natural language processing through this comprehensive lecture. Delve into the differences between discriminative and generative models, understanding the importance of latent random variables in NLP tasks. Learn about the Variational Autoencoder (VAE) objective, its interpretation, and solutions to sampling issues. Discover techniques for generating language models with latent variables, including KL divergence annealing and weakening the decoder. Examine methods for handling discrete latent variables, such as enumeration and the Gumbel-Softmax technique. Investigate practical applications in controllable text generation and symbol sequence latent variables. Gain insights into the challenges and solutions in training these models, equipping yourself with advanced knowledge in neural network approaches for NLP.

CMU Neural Nets for NLP - Models with Latent Random Variables

Graham Neubig
Add to list