Examples of Pre-training Encoders . Common to pre-train encoders for downstream tasks, common to use
9
Regularization for Pre-training (e.g. Barone et al. 2017) Pre-training relies on the fact that we won't move too far from the
10
Selective Parameter Adaptation Sometimes it is better to adapt only some of the parameters
11
Soft Parameter Tying
12
Supervised Domain Adaptation through Feature Augmentation
13
Unsupervised Learning through Feature Matching
14
Multilingual Structured Prediction/ Multilingual Outputs • Things are harder when predicting a sequence of actions (parsing) or words (MT) in different languages
15
Multi-lingual Sequence-to- sequence Models
16
Types of Multi-tasking
17
Multiple Annotation Standards
18
Different Layers for Different
19
Summary of design dimensions
Description:
Explore multi-task and multi-lingual learning in natural language processing through this comprehensive lecture from CMU's Neural Networks for NLP course. Delve into key concepts like feature extraction, increasing data through multi-tasking, pre-training encoders, and regularization techniques. Examine various approaches to multi-task learning, including selective parameter adaptation, soft parameter tying, and supervised domain adaptation. Investigate multilingual structured prediction, sequence-to-sequence models, and the challenges of handling multiple annotation standards. Gain insights into designing effective multi-task and multi-lingual NLP systems through practical examples and design principles presented by Antonis Anastasopoulos.
Neural Nets for NLP - Multi-task, Multi-lingual Learning