Domain Specific Architectures for Deep Neural Networks: Three Generations of Tensor Processing Units
Explore the evolution of Domain Specific Architectures for Deep Neural Networks through three generations of Tensor Processing Units (TPUs) in this distinguished lecture by David Patterson. Delve into the resurgence of domain-specific architectures driven by the success of deep neural networks and the slowing of Moore's Law. Learn about Google's TPUv1 and its significant performance improvements for inference tasks. Discover how Google developed the first production DSA supercomputer for the more challenging task of training, deployed in 2017. Examine the impressive capabilities of TPUv2 and TPUv3 supercomputers in training production DNNs with near-perfect linear speedup and superior energy efficiency compared to general-purpose supercomputers. Gain insights from Patterson's extensive experience in computer architecture, including his work on RISC, RAID, and Network of Workstation projects, which have had a profound impact on the industry.