Apache Kafka and Kubernetes in the Telco Industry for Cloud-Native 5G, MEC and OSS Use Cases
Explore cloud-native solutions for 5G, MEC, and OSS in telecommunications using Apache Kafka and Kubernetes, focusing on data motion and infrastructure modernization.
Querying Live Data Using Open Source SQL Engines - A Comparison of Apache Flink, ksqlDB, and Other Tools
Explore real-time data querying using SQL engines like Apache Flink, ksqlDB, Trino, and RisingWave for streaming data analysis on Kafka and other message buses.
Growing the Delta Ecosystem - Delta-RS for Rust and Python
Explore delta-rs, a Rust library extending Delta Lake beyond Spark. Learn its current capabilities, roadmap, and real-world application in kafka-delta-ingest for high-throughput data transfer.
Automating FAA's SWIM Data Ingestion and Analysis with Databricks
Automating FAA's SWIM data ingestion and analysis using cloud-based services, showcasing infrastructure setup, configuration, and insights derived from flight data for improved air transportation management.
Hail Hydrate - From Stream to Lake Using Open Source
Explore open-source tools for efficient data streaming and lake management, enhancing your skills in modern data architecture and processing techniques.
Astronomer's Journey from Cloud Lock-in to Open Source Independence
Discover how Astronomer transitioned from AWS-managed services to open-source solutions, enhancing flexibility and independence in their data engineering platform.
Building Robust Streaming Data Pipelines with Apache Spark
Learn to build robust streaming data pipelines using Apache Spark, Kafka, and Camel. Explore ETL challenges, data management, and lessons from running these technologies in Docker.
Logging Modernization and Cybersecurity at Scale with MiNiFi, Kafka and Flink
Modernize logging and enhance cybersecurity using MiNiFi, Kafka, and Flink. Learn efficient log collection, processing, and streaming analytics for global network monitoring and alert generation.
Building a Real-Time Analytics Database
Odkryj tajniki tworzenia bazy danych do analityki w czasie rzeczywistym. Poznaj kluczowe koncepcje i najlepsze praktyki w tej dziedzinie.
Evolution of Real-time User Engagement Event Consumption at Pinterest - Current 2023
Explore Pinterest's evolution in real-time event processing using Apache Flink, improving data quality and efficiency while saving millions in infrastructure costs and boosting user engagement.
Data Pipelines Evolution from Batch to Streaming
Explore the evolution from batch to streaming data pipelines using Apache Flink and Kafka. Learn strategies for seamless transition, including query-based connectors, change data capture, and handling late events.
Streaming vs. Batch Processing: The Future of Data Architecture
Explore the future of data processing with industry experts as they debate streaming vs. batch, adoption challenges, and best practices for real-time data architecture.
Producing Apache Kafka Messages Using Apache Flink and Java
Learn to produce Apache Kafka messages using Apache Flink and Java in this comprehensive walkthrough. Covers Flink job setup, Kafka configuration, data generation, serialization, and execution verification.
Event Time and Watermarks in Apache Flink
Explore event time, processing time, and watermarks in Apache Flink for effective stream processing and window operations.
Building Realtime Data Warehouses from Scratch - End-to-End Data Engineering Project
Design and implement a real-time data warehouse with Apache Airflow, Kafka, and Pinot. Create custom hooks, ingest batch and streaming data, and visualize evolving data using Apache Superset.
Understanding Stream-Table Duality Using Kafka, Flink SQL and Debezium Format
Explore stream-table duality using Kafka, Flink SQL, and Debezium format. Learn about CDC events, SQL queries on changelog data, and practical applications through a demo.
Environmental Sustainability: An Energy-Aware Datacenter Operating System
Explore an energy-aware datacenter operating system that optimizes power consumption in distributed computing environments using open-source software, machine learning, and Kubernetes-based workload optimization.
Unlocking Financial Data with Real-Time Pipelines Using Apache Technologies
Discover how to build powerful real-time data pipelines using Apache Flink, NiFi, Kafka, and Iceberg for processing financial data efficiently, enabling swift decision-making and regulatory compliance.
Reinventing Kafka in the Data Streaming Era
Explore Apache Kafka's evolution in modern data streaming, from core architecture to cloud-native innovations and exactly-once semantics for real-time enterprise applications.
A Journey of Audit Logs from Kafka to Elasticsearch - Building a Centralized Logging Platform
Discover how to build a scalable audit logging platform using Kafka and OpenSearch, focusing on tracking data modifications across multiple users and data models in business-critical systems.
Practices of Kafka Cloud-Native Architecture
Discover how XiaoHongShu implements Kafka cloud-native architecture, exploring real-world practices, data storage solutions, and messaging engine optimization in large-scale deployments.