Explore the intricate architecture behind Twitter's PowerTrack, a product designed for Enterprise customers to filter and receive customized subsets of Tweets in real-time. Delve into the technical challenges and solutions for filtering data at scale, ensuring application and system layer redundancy, applying configuration changes to open streaming connections, backfilling data for 100% fidelity, and managing customer stream states across data centers. Learn from Lisa White, tech lead of Twitter's streaming data products team, as she shares insights on the complex process of delivering millions of daily Tweets to hundreds of customers with thousands of end-users without missing a single Tweet. Gain a comprehensive understanding of the PowerTrack system, including its public firehose, customer rule predicate indexing, rule evaluation, index architecture, messaging queue, filtering layer, rules table, and state management. This conference talk from Strange Loop offers a deep dive into the technical intricacies of large-scale, real-time data delivery systems. Read more