LLMOps: OpenVino Toolkit para Quantizar LLama 3.2 3B a 4int e Inferencia en CPU

LLMOps: OpenVino Toolkit para Quantizar LLama 3.2 3B a 4int e Inferencia en CPU

Explore a 32-minute video tutorial on LLMOps, focusing on using the OpenVino Toolkit to quantize the LLAMA3.2 3B model to 4-bit integer format and perform CPU inference. Learn how to convert the LLAMA3.2 3 billion parameter model to OpenVino IR format, apply 4-bit integer quantization, and execute inference on CPU using Chain of Thought (CoT) prompts. Access the accompanying Jupyter notebook for hands-on practice and in-depth understanding of the process. Ideal for data scientists and machine learning enthusiasts looking to optimize large language models for efficient deployment.