Optimizing Amazon EventBridge Pipes with Apache Kafka for ProSiebenSat.1
Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only! Grab it Explore the optimization of Amazon EventBridge Pipes with Apache Kafka in this conference talk from GOTO EDA Day 2024. Learn how ProSiebenSat.1 Media SE tackled the challenge of serving millions of customers during major events for their streaming application, Joyn. Discover the journey of optimizing Kafka within their system, including the decision to use Confluent Kafka over Amazon MSK. Gain insights into integrating Kafka with Amazon EventBridge Pipes, addressing limitations and issues encountered. Examine architectural patterns like Claim-check and Point-to-point fan-out for enhancing resilience and efficiency. Understand the speakers' wishlist for AWS improvements in payload size, monitoring capabilities, and processing speed. Conclude with valuable lessons learned and strategies for effectively leveraging Kafka in complex systems.