Fine-tuning Pixtral - Multi-modal Vision and Text Model
Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only! Grab it Explore the process of fine-tuning Pixtral, a multi-modal vision and text model, in this comprehensive tutorial video. Learn about Pixtral's architecture, including its custom image encoder trained from scratch, and follow step-by-step instructions for fine-tuning in a Jupyter notebook. Discover GPU setup requirements, dataset preparation techniques, and advanced chat templating. Gain insights into evaluating baseline performance, setting up LoRA fine-tuning, and optimizing training arguments. Explore methods for merging LoRA adapters, measuring OCR performance, and setting up an API endpoint using vLLM for inference. Access additional resources, including slides, datasets, and code repositories, to enhance your understanding of Pixtral fine-tuning techniques.