Pushing the Limits of RNN Compression Using Kronecker Products
Explore the cutting-edge techniques for compressing Recurrent Neural Networks (RNNs) using Kronecker Products in this tinyML Talks webcast. Delve into a two-part presentation that first demonstrates how Kronecker Products can achieve 15-38x compression factors for IoT RNN applications, outperforming traditional compression methods. Learn the fundamentals of Kronecker Products and best practices for compressing IoT workloads. Then, discover a novel technique called "doping" that addresses accuracy loss in large Natural Language Processing tasks when applying Kronecker Product compression. Understand the concept of Doped Kronecker Product (DKP) and its potential to recover lost accuracy. Gain insights into the power of Kronecker Products, their implementation on existing hardware, and their practical applications in various domains including chatbots.