Processing Fast Data with Apache Spark - The Tale of Two Streaming APIs
Explore the world of fast data processing with Apache Spark in this 50-minute conference talk from Devoxx. Dive into the two streaming APIs offered by Spark: the low-level Spark Streaming and the more declarative Structured Streaming. Learn about their capabilities, key differences, and practical applications in enterprise-level data analytics. Discover how to get started with both APIs, handle time and state management, implement functional coding strategies, and leverage machine learning capabilities. Through real-world examples, gain insights on choosing the right API or combining both to create functional and resilient streaming pipelines. Benefit from the expertise of Gerard Maas, a senior software engineer at Lightbend and co-author of "Stream Processing with Apache Spark," as he guides you through the intricacies of fast data architectures and their role in accelerating decision-making processes for businesses.