SLA-Aware Batch Scheduling in Apache YuniKorn with Multi-Tenant Preemption
Explore multi-tenant batch scheduling in Kubernetes with Apache YuniKorn in this 25-minute conference talk. Dive deep into the challenges of resource-intensive data engineering use cases and learn how YuniKorn addresses the limitations of the default scheduler's priority-based approach. Discover the concept of "The Laws of Preemption" for batch processing and understand how to implement flexible, context-aware preemption that considers workload types, user quotas, and hierarchical queues. Gain insights into preventing preemption storms, defining guardrails for cross-tenant resource allocation, and implementing tenant-aware preemption based on YuniKorn's hierarchical queue model and guaranteed resources. Enhance your knowledge of advanced scheduling techniques for optimizing multi-tenant environments in Kubernetes clusters.