Stream All Things - Patterns of Modern Data Integration
Explore patterns of modern data integration in this conference talk from Philly ETE 2017. Delve into the challenges of data integration and learn about design and architecture patterns used to address these issues. Discover how Apache Kafka can be implemented to create fast, scalable, and manageable data pipelines. Examine topics such as streams of events, Kafka architecture, timestamp architecture, schema registry, Kafka scaling, and Kafka Connect. Gain insights into enriching events, basic architecture, latency considerations, and caching strategies. Investigate advanced concepts like search relevance, window joints, time updates, and buffers. Conclude with a recap of Kafka's applications in New York City and participate in a Q&A session.