How to Survive High Demanding Services - Managing AI with Kubernetes and Istio

How to Survive High Demanding Services - Managing AI with Kubernetes and Istio

Explore strategies for managing high-demand artificial intelligence services in this 34-minute conference talk by Endika Gandarias from the Basque Government Informatic Society. Delve into the challenges of handling hundreds of thousands of daily requests and learn effective design principles for maintaining uninterrupted service. Discover techniques for real-time health status monitoring, automated alarm management, artifact version evaluation under real load conditions, abuse prevention, and auto-recovery. Gain insights into the Kubernetes and Istio service mesh ecosystem through an in-depth examination of the Itzuli language tools project, showcasing practical decisions made to ensure robust and scalable AI service management.