High-Performance AI Model Serving with Ray Serve - A Rubrik Case Study
Explore a technical conference talk from Ray Summit 2024 where Rubrik engineers Shaikh Ismail and Shivanshu Agrawal demonstrate how they leveraged Ray Serve to achieve high-performance AI inference at scale. Discover the technical journey of implementing Ray's ML model serving library to handle millions of daily evaluations while meeting demanding scalability and throughput requirements. Learn about the distinctive features that made Ray Serve the optimal choice for online inference scenarios, and gain practical insights into addressing critical challenges including fault tolerance, robustness, and Kubernetes deployment. Gain valuable knowledge applicable to organizations seeking to enhance their AI serving infrastructure for high-stakes, real-time applications.