Efficient Deep Learning Training with Ludwig AutoML, Ray, and Nodeless Kubernetes
Explore efficient deep learning training techniques using Ludwig AutoML, Ray, and Nodeless Kubernetes in this informative conference talk. Discover how open-source platforms Ray and Ludwig make deep learning more accessible by reducing complexity barriers in training, scaling, deploying, and serving models. Learn about the challenges of deep learning's cost and operational overhead, particularly in managing GPU resources for model development, testing, and tuning. Understand the benefits of running Ray and Ludwig on cloud Kubernetes clusters, utilizing Nodeless Kubernetes to dynamically add and remove right-sized GPU resources as needed. Examine experimental results comparing the cost and operational efficiency of using Nodeless Kubernetes versus direct EC2 deployment, revealing significant improvements in both efficiency and usability for deep learning workflows.