Automated Monitoring in Production for Continuous Model Improvements
Discover how to implement automated monitoring for machine learning models in production to ensure continuous improvements. Learn about the critical importance of model monitoring in MLOps and why it's often neglected. Explore a core machine learning service that provides automated, continuous evaluation of deployed model performance using metrics like AUC and RMSE. Gain insights into visualizing model output, detecting performance degradation, and attributing problems to input data characteristics. Understand the components required to build such a service, including a scalable backend using Spark on Azure Databricks, REST endpoints powered by Python-Flask, and a React-based UI. Learn how to slice metrics by input features for deeper insights and potential model improvements. Benefit from the speakers' extensive experience in developing scalable machine learning services and products at Adobe.