Bring Batch Capability Into Kubernetes, Using AI and Big Data as an Example
Explore a 23-minute conference talk that delves into bringing batch capabilities to Kubernetes, with a focus on AI and Big Data applications. Learn how Kubernetes, originally designed for serving jobs, is evolving to accommodate batch workloads like Spark and TensorFlow in multi-tenant environments. Discover advanced scheduling features such as fair-share scheduling and how they benefit various batch and elastic workload classes, including machine learning, deep learning, bioinformatics, genomics, and other big data applications. Gain insights into the integration of generalized domain frameworks like TensorFlow, Spark, PyTorch, and MPI with Volcano, a new CNCF project. Understand the features Volcano provides, examine concrete examples, and learn about next steps and opportunities for involvement in this emerging technology.