Главная
Study mode:
on
1
Intro
2
Phi3 mini 128k
3
Defining input and output parameters
4
Defining artifacts
5
Flowing the model
6
MLFlow notebook
7
MLFlow model
8
ONNX model
9
ONNX performance
10
DirectML
11
Microsoft ONNX
12
Conclusion
Description:
Explore the Microsoft Phi3 Mini 128k model and compare inference performance across different formats and quantization methods in this 45-minute video tutorial. Learn how to work with MLFlow, GGUF, and ONNX formats while examining their impact on inference time and precision. Follow along with provided notebooks to implement MLFlow quantization with bfloat16, Llama.cpp quantization with float16 in GGUF format, ONNX CPU quantization with int4, and ONNX GPU DirectML quantization with int4. Gain insights into defining input and output parameters, managing artifacts, and flowing the model through various frameworks. Conclude with a comprehensive understanding of the performance differences between these approaches for deploying the Phi3 mini 128k model in machine learning and data science applications.

MLOps: Comparing Microsoft Phi3 Mini 128k in GGUF, MLFlow, and ONNX Formats

The Machine Learning Engineer
Add to list
0:00 / 0:00