A Simple Example • How long does a metro-matrix multiply take?
4
Practically
5
Speed Trick 3
6
Reduce # of Operations
7
Reduce CPU-GPU Data Movement
8
What About Memory?
9
Three Types of Parallelism
10
Within-operation Parallelism
11
Operation-wise Parallelism
12
Example-wise Parallelism
13
Computation Across Large Vocabularies
14
A Visual Example of the Softmax
15
Importance Sampling (Bengio and Senecal 2003)
16
Noise Contrastive Estimation (Mnih & Teh 2012)
17
Mini-batch Based Negative Sampling
18
Hard Negative Mining • Select the top n hardest examples
19
Efficient Maximum Inner Product Search
20
Structure-based Approximations
21
Class-based Softmax (Goodman 2001) • Assign each word to a class
22
Binary Code Prediction (Dietterich and Bakiri 1995, Oda et al. 2017)
23
Two Improvement to Binary Code Prediction
Description:
Learn efficiency tricks for neural networks in natural language processing through this lecture from CMU's Neural Networks for NLP course. Explore GPU training tips, parallel training techniques, negative sampling methods, and softmax approximations like hierarchical softmax. Discover practical strategies to reduce operations, optimize CPU-GPU data movement, and manage memory effectively. Dive into various parallelism types, tackle computation challenges with large vocabularies, and examine advanced techniques like importance sampling, noise contrastive estimation, and efficient maximum inner product search. Gain insights into structure-based approximations, including class-based softmax and binary code prediction, to enhance neural network performance in NLP tasks.
Neural Nets for NLP - Efficiency Tricks for Neural Nets