Scenario: Big scale cluster: 10k nodes, 1 million Pod
19
Use Case: Al platform at Xiaohongshu
20
Use Case: Batch at Ruitian investment
21
Community
22
Release
Description:
Explore a comprehensive introduction and deep dive into Volcano, a system for running high-performance workloads on Kubernetes, in this 32-minute conference talk by Klaus Ma from Huawei Cloud. Discover how Volcano addresses the challenges of batch scheduling in Kubernetes, providing powerful capabilities for ML/DL, big data applications, and Bioinformatics/Genomics. Learn about key concepts such as job management, resource management with queues, and dynamic resource sharing. Delve into various scheduling scenarios, including elastic scheduling, topology awareness, SLA scheduling, and CPU topology awareness. Gain insights into Volcano's application in batch scheduling for Spark, co-location and oversubscription, and its ability to handle large-scale clusters. Examine real-world use cases from Xiaohongshu's AI platform and Ruitian Investment's batch processing. Understand the project's community involvement and release cycle, equipping yourself with valuable knowledge for running high-performance workloads in Kubernetes environments.
Read more
Volcano: Introduction and Deep Dive into High-Performance Workload Management on Kubernetes