Data in Motion - Streaming Static Data Efficiently in Akka Persistence
Explore the advantages and concepts of streaming data processing in this conference talk from Scala Days New York 2016. Dive into the world of Akka Persistence Query and its implementation in the Cassandra plugin for Akka Persistence. Learn about architecture and design considerations, implementation details, and performance tuning for distributed systems. Discover how to handle static data as streaming data sources, and understand the differences between batch processing and data in motion. Examine the intricacies of pulling data from sources, including inserts and updates, as well as pushing data from infinite streams of finite data sources. Gain insights into log data structures, actor publishers, and event processing by persistence ID and tags. Explore non-blocking asynchronous replay, benchmarks, and alternative architectures for modern reactive enterprise stream processing applications. Address challenges in distributed systems, such as ordering, causal stream merging, and exactly-once delivery optimization. Conclude with a discussion on the applications of infinite streams and the future of distributed systems in data processing. Read more