Scale Your Batch, Big Data, and AI Workloads Beyond the Kubernetes Scheduler

Scale Your Batch, Big Data, and AI Workloads Beyond the Kubernetes Scheduler

Explore the challenges and solutions for scaling batch, big data, and AI workloads on Kubernetes in this 25-minute DevConf.US 2024 conference talk. Delve into the limitations of the traditional Kubernetes scheduler when handling resource-intensive, highly-coupled processes. Discover recent innovations in the Kubernetes ecosystem designed to address issues such as resource fragmentation, quota management, auto-scaling, and priority handling. Compare projects like Karmada, Koordinator, Kueue, MCAD, Volcano, and YuniKorn, examining their design choices and trade-offs. Gain insights to help you select the most suitable solution for optimizing cluster utilization and managing batch workloads effectively in your Kubernetes environment.