Load Shedding for Evil Wizards
Explore load shedding mechanisms in Envoy through this 32-minute conference talk from KubeCon + CloudNativeCon Europe 2022. Dive deep into circuit breaking, rate limiting, admission control, and adaptive concurrency, understanding their implementations, use cases, and potential pitfalls. Gain insights into queuing and scheduling concepts behind these mechanisms, and learn how they can fail to protect against noisy-neighbor problems. Discover experimental results based on FBThrift work and explore potential community improvements for Envoy. Examine topics such as Little's Law applied to servers, simulating overload, rate/bandwidth limits vs. concurrency control, shortcomings of circuit breakers, and the adaptive concurrency control filter. Understand the gradient, concurrency limit behavior, configuration of adaptive concurrency, and its drawbacks, as well as the admission control filter.