Fine-tuning Multi-modal LLaVA Vision and Language Models
Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only! Grab it Learn how to fine-tune multi-modal vision and language models like LLaVA in this comprehensive tutorial. Explore the architectures of LLaVA 1.5, LLaVA 1.6, and IDEFICS, and understand their applications compared to ChatGPT. Dive into the intricacies of vision encoder architecture and multi-modal model design. Master the process of data creation, dataset preparation, and fine-tuning techniques. Gain hands-on experience with data loading, LoRA setup, and evaluation methods. Follow along with practical demonstrations on training, inference, and post-training evaluation. Clarify technical concepts and summarize key takeaways to enhance your skills in working with advanced vision and language models.