Explore a groundbreaking approach to running neural networks on resource-constrained devices in this keynote address from the tinyML Summit 2021. Delve into the concept of adaptive neural networks that dynamically minimize memory and computational requirements during inference. Learn about the challenges facing tinyML, the basics of dynamic inference, and its relationship with hardware. Discover throttleable neural networks (TNN) and their intuitions, controller training techniques, and early results on hardware. Examine practical applications through object detection examples, metrics for agility, and development workflows. Gain insights into hardware accelerators and how this adaptive approach enables more flexible and efficient deployment of machine learning models on tiny devices.