Explore the intricacies of testing recommender systems in real-world scenarios through this comprehensive conference talk from MLOps World: Machine Learning in Production. Delve into the limitations of traditional evaluation metrics and discover the importance of behavioral testing for recommender systems. Learn how to leverage RecList, an open-source package, to scale up real-world testing in both research and production environments. Gain hands-on experience through practical coding examples and demonstrations. Understand the nuances of ad hoc error analysis and its role in ensuring desired quality in the wild. Join Jacopo Tagliabue, Director of AI at Coveo and Adjunct Professor of MLSys at NYU, as he shares insights from his extensive experience in shipping models to hundreds of customers and millions of users.