Is Your GPU Really Working Efficiently in the Data Center? N Ways to Improve GPU Usage

Is Your GPU Really Working Efficiently in the Data Center? N Ways to Improve GPU Usage

Explore strategies to optimize GPU efficiency in data centers through this informative conference talk. Learn about improving Model Flops Utilization (MFU) for AI accelerators, including GPUs and NPUs, in large-scale Kubernetes clusters. Discover techniques for training Large Language Models (LLMs) with billions of parameters, such as model parallelism, switch-affinity scheduling, and checkpoint efficiency optimization. Gain insights into GPU sharing technology, training-inference hybrid solutions for tidal scenarios, and node grouping methods to enhance GPU utilization. Understand how to assess GPU performance, address monopolization by underutilized applications, and ensure 24/7 efficiency of AI devices in various industries.