Главная
Study mode:
on
1
Introduction
2
Unitys Goal
3
Parallelization
4
Parallel Computation Graph
5
Data Parallelization
6
PCG Advantages
7
Techniques
8
Results
9
Conclusion
Description:
Explore a conference talk from OSDI '22 that introduces Unity, a groundbreaking system for optimizing distributed Deep Neural Network (DNN) training. Delve into how Unity jointly optimizes algebraic transformations and parallelization using a unified parallel computation graph (PCG). Learn about the system's innovative approach to automatically generating and verifying optimizations, as well as its hierarchical search algorithm for maintaining scalability. Discover Unity's performance improvements over existing DNN training frameworks, with evaluations conducted on seven real-world DNNs using up to 192 GPUs across 32 nodes. Gain insights into the potential impact of Unity on accelerating DNN training and its availability as part of the open-source FlexFlow framework.

Unity - Accelerating DNN Training Through Joint Optimization of Algebraic Transformations and Parallelization

USENIX
Add to list
00:00
-00:24