Using Serverless and ARM64 for Real-Time Observability
Explore how a sub-second latency query engine inspired by Facebook's Scuba was extended to handle massive data volumes using cloud object storage and serverless functions in this conference talk from GOTO Copenhagen 2023. Dive into the challenges of managing AWS Lambda at scale, including issues with impatience, maximum concurrency, runtime and hardware architecture configuration, and the price/performance trade-offs of using thousands of parallel workers. Learn about interactive investigation of production behavior, dynamic aggregation across any time range, and the benefits of buying compute in 1ms units. Gain insights into considerations for implementing serverless and ARM64 architectures for real-time observability, and understand why it's not just about cost savings. Discover practical tips and best practices for applying these concepts in your own projects.