Real-Time Data Streaming Architectures for Generative AI
Explore the evolution of data processing architectures for Generative AI in this 12-minute talk by Emily Ekdahl at the MLOps.community event. Discover how real-time data streaming solutions using Apache Kafka and Apache Flink are revolutionizing the way…
Jingdong Algorithm Debug System - StarGazing Introduction
Discover how to effectively debug algorithms using Jingdong's StarGazing system, exploring its features and implementation in large-scale AI development environments.
Analysis and Application of New Features in Flink ML 2.2.0
Explore Flink ML 2.2.0's new features, including online inference, feature engineering, and real-time log clustering applications in Alibaba's infrastructure.
Construction Practice of Xiaomi Flink Real-Time Computing Platform
Explore Xiaomi's real-time computing platform construction, focusing on resource flexibility, cost-effectiveness, and usability. Gain insights into architecture, metadata management, and operational governance.
FlinkSQL's Field Lineage and Data Permission Solution
Explore FlinkSQL's field lineage and data permission solutions for real-time data warehouses. Learn to implement Atlas and Ranger-like capabilities without source code intrusion.
Apache Flink Stream Batch Adaptive Shuffle - Latest Progress and Future Plans
Explore Flink's latest Shuffle 3.0 architecture, featuring cloud-native design, stream-batch fusion, and adaptive capabilities for enhanced resource management and performance.
Apache Paimon - A Unified Data Lake for Stream, Batch, and OLAP Processing
Explore Apache Paimon's capabilities as a unified data lake system, integrating stream processing, batch operations, and OLAP functionalities for modern data architecture.
Querying Live Data Using Open Source SQL Engines - A Comparison of Apache Flink, ksqlDB, and Other Tools
Explore real-time data querying using SQL engines like Apache Flink, ksqlDB, Trino, and RisingWave for streaming data analysis on Kafka and other message buses.
Transforming Streams with Apache Flink - From Click to Insight
Master real-time data processing with Apache Flink, from implementing counters and windowed analytics to data enrichment techniques for high-demand distributed systems.
Real-Time Analytics of Customer Financial Activities with Apache Flink
Explore real-time analytics for financial activities using Apache Flink. Learn about Capital One's approach to processing customer data, event-time processing, and applications in fraud detection.
Delta Lake 2.0 Overview - New Features and Community Collaborations
Overview of Delta Lake 2.0 features, integrations, and community collaboration. Highlights include data skipping, Change Data Feed, column mapping, multi-cluster S3 writes, and support for various languages and platforms.
Delta Lake Roadmap 2021 H2 - Features Overview and Community Feedback
Overview of Delta Lake's 2021 roadmap, highlighting new features, integrations, and community-driven developments for enhanced data management and analytics capabilities.
Introduction to Stream Processing with Apache Flink
Explore stream processing fundamentals and Apache Flink's capabilities for real-time data analysis, scalable applications, and event-driven architectures.
Hail Hydrate - From Stream to Lake Using Open Source
Explore open-source tools for efficient data streaming and lake management, enhancing your skills in modern data architecture and processing techniques.
Through the Looking Glass: Key Architectural Choices in Flink and Kafka Streams
Explore architectural differences between Apache Flink and Kafka Streams, gaining insights into stream processing fundamentals and design choices for improved system performance.
Logging Modernization and Cybersecurity at Scale with MiNiFi, Kafka and Flink
Modernize logging and enhance cybersecurity using MiNiFi, Kafka, and Flink. Learn efficient log collection, processing, and streaming analytics for global network monitoring and alert generation.
Practice of Building Real-Time Data Lake Based on Flink
Build a real-time data lake using Flink and Iceberg to address timeliness, diversity, consistency, and accuracy challenges in modern data architectures.
Challenges and Solutions for Building Real-time Data Warehousing with Apache Flink, Apache Hive and Apache Iceberg
Explore solutions for real-time data warehousing using Apache Flink, Hive, and Iceberg. Learn to handle challenges like late events and dirty data routing in enterprise-level ETL migration.
State of Scala API in Apache Flink
Explore Scala API in Apache Flink 1.15+, including Scala 3 support, SBT project setup, and community wrappers. Learn to leverage Scala for efficient Flink job development and packaging.
Flink K8S Operator Autoscaling - Implementation and Best Practices
Explore Flink K8S Operator's autoscaling capabilities, origins, and future developments. Learn about resource optimization and non-stop updates in Apache Flink's stream processing ecosystem.
Real-Time Data Integration Practice Based on Flink CDC at Alibaba Cloud
Explore real-time data integration using Flink CDC, covering core design, key implementations, and Alibaba Cloud's solutions for data lake and warehouse integration scenarios.