Главная
Study mode:
on
1
- Intro
2
- Sponsor: Fully Connected by Weights & Biases
3
- Paper Overview
4
- What's the problem with sampling?
5
- Beam Search: The good and the bad
6
- Top-k and Nucleus Sampling
7
- Why the most likely things might not be the best
8
- The expected information content of the next word
9
- How to trade off information and likelihood
10
- Connections to information theory and psycholinguistics
11
- Introducing Typical Sampling
12
- Experimental Evaluation
13
- My thoughts on this paper
Description:
Explore the concept of typical decoding for natural language generation in this 49-minute video lecture. Learn about the challenges of generating human-like text from language models and discover a new decoding method called typical sampling. Understand the trade-off between high-probability and high-information samples, and how this approach connects to psycholinguistic theories of human speech generation. Examine the limitations of current sampling methods like top-k and nucleus sampling, and see how typical sampling offers a more principled and effective alternative. Follow along as the video breaks down the paper's key ideas, experimental results, and potential implications for improving text generation from AI language models.

Typical Decoding for Natural Language Generation - Get More Human-Like Outputs From Language Models

Yannic Kilcher
Add to list
0:00 / 0:00