Deployment - FSDL 2022
Explore the process of transforming a promising machine learning model into a valuable ML-powered product in this comprehensive lecture. Learn about various deployment architectures, including model-in-server, model-in-database, and model-as-a-service. Discover how to create prototypes using tools like Gradio and Streamlit, implement REST APIs, manage dependencies, and containerize services with Docker. Delve into performance optimization techniques for both CPUs and GPUs, including distillation, quantization, caching, and batching. Examine horizontal scaling strategies, container orchestration with Kubernetes, and serverless options. Investigate rollout techniques such as shadow and canary deployments, and explore managed services like AWS SageMaker. Finally, gain insights into edge deployment, efficient model creation for edge devices, and key takeaways for successfully deploying ML models in various environments.