Главная
Study mode:
on
1
Welcome
2
Mistral v0.3
3
Phi-3 models
4
Full fine-tuning
5
LoRA
6
Picking LoRA alpha and rank
7
Running training with SFTTrainer from TRL
Description:
Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only! Grab it Dive into a comprehensive video tutorial on LoRA fine-tuning for machine learning models. Explore recent developments in Mistral v0.3 and Phi-3 models before delving into full fine-tuning techniques. Learn the intricacies of LoRA, including how to select optimal alpha and rank parameters. Discover strategies for choosing fine-tuning parameters such as learning rate, schedule, and batch size. Gain insights into advanced optimizations like rank stabilized LoRA, loftQ, and LoRA+. Follow along with a practical demonstration using SFTTrainer from TRL to run training sessions. Access additional resources and support, as well as a companion notebook, to enhance your understanding of LoRA fine-tuning techniques.

LoRA Fine-tuning Explained - Choosing Parameters and Optimizations

Trelis Research
Add to list
0:00 / 0:00