Benchmarking - You're Doing It Wrong
Discover the art of effective benchmarking in this 35-minute conference talk from Strange Loop. Explore common pitfalls and biases that can invalidate performance measurements, and learn essential techniques for creating accurate and useful benchmarks. Delve into real-world examples of flawed benchmarks, understand the impact of caches on performance testing, and gain insights into multimodal distributions. Examine the importance of focusing on critical program components and avoiding premature optimization. Uncover the challenges of microbenchmarking, non-constant work per iteration, and outlier management. Acquire valuable skills to guide your team towards meaningful performance improvements and avoid potential setbacks in system optimization.