Building Data Infrastructure at Scale for AI/ML with Open Data Lakehouses
Explore how data lakehouse architecture with Apache Hudi supports real-world predictive ML and vector-based AI use cases in this 30-minute keynote by Vinoth Chandar, creator of Apache Hudi. Learn about ingesting data with minute-level freshness, providin…
Data Streaming with Confluent: Stream, Connect, Govern, and Process Information
Explore the world of data streaming through this 39-minute conference talk from Devoxx. Dive into Confluent's Data Streaming Platform and discover how to Stream, Connect, Govern, and Process large volumes of information effectively. Learn about the revol…
Building Real-Time ML Features with a Feature Platform
Explore challenges in deploying ML pipelines and learn how feature stores and platforms solve data problems for production ML, enabling real-time processing and scaling.
Apache Spark Full Course - Learn Apache Spark 2020
In this 'Spark Tutorial' you will comprehensively learn all the major concepts of Spark such as Spark RDD, Dataframes, Spark SQL and Spark Streaming.
Apache Kafka Tutorials
Comprehensive tutorials on Apache Kafka, covering core concepts, installation, APIs, configurations, and advanced topics for building real-time data streaming applications.
Streaming Featurization with Ibis, Substrait and Apache Arrow
Explore real-time streaming data processing using Ibis, Substrait, and Apache Arrow. Learn how this software stack enhances featurization workflows for fast, accurate insights and decision-making in data science.
Delta Lake Streaming - Internals and Query Progress Logs
Explore Delta Lake streaming internals, focusing on structured streaming components, Query Progress Logs, and checkpoint directories for efficient real-time data pipeline management.
Designing and Implementing a Real-time Data Lake with Dynamically Changing Schema
Learn to build a robust streaming ETL pipeline that handles dynamically changing schemas and new event types without downtime, using Delta Lake and Databricks for real-time data lake implementation.
SIGCOMM 2020 - Reducto - On-Camera Filtering for Resource-Efficient Real-Time Video Analytics
Explore real-time video analytics optimization with Reducto, a filtering technique for efficient on-camera processing. Learn about system design, performance results, and practical applications.
Building a Real-Time Feature Store for Machine Learning at iFood
Discover how iFood built a real-time feature store using Databricks and Spark Structured Streaming to process event streams, store data, and develop reliable code for efficient machine learning model predictions.
Real-time Trade Data Processing and Machine Learning at Giga-Scale with Hazelcast and FINOS Perspective
Explore scalable real-time trade data processing using Hazelcast and FINOS Perspective. Learn to architect high-performance applications combining streaming data, reference data, and machine learning for front and back office use-cases.
Building an Open Source Streaming Analytics Stack with Kafka and Druid
Learn to build a robust streaming analytics stack using Kafka and Druid for real-time data processing, flexible querying, and maintaining data integrity in high-volume environments.
Introduction to Fluvio - The Programmable Platform for Data in Motion
Explore Fluvio, a programmable platform revolutionizing data-in-motion processing. Learn its key features and applications in modern data architectures for efficient, real-time data handling.
Interactive Brokers API and TradingView Charts in Python
Learn to integrate TradingView Lightweight Charts with Interactive Brokers API in Python. Build projects for data visualization, real-time updates, and automated trading with hands-on coding examples.
Productionizing Big Data - Stories from the Trenches
Uncover mysteries of data pipeline anomalies, learn approaches to fixing and preventing issues, and gain insights into handling real-world production challenges in big data and AdTech.
Evolution of Real-time User Engagement Event Consumption at Pinterest - Current 2023
Explore Pinterest's evolution in real-time event processing using Apache Flink, improving data quality and efficiency while saving millions in infrastructure costs and boosting user engagement.
Realtime Stock Market Anomaly Detection Using Machine Learning - End-to-End Data Engineering Project
Develop an end-to-end data engineering project for real-time stock market anomaly detection using Quix Streams, Redpanda, and Docker. Learn to build a complete data pipeline and deploy an advanced ML model.
Earthquake Early Warning Capabilities and Delivery Around the World - Part 2
Explore global earthquake early warning systems, their capabilities, and delivery methods in this comprehensive session from the AGU Fall Meeting 2011.
Insight of Connected Compute in Vehicles
Explore connected computing in vehicles with insights from industry expert Masa Yoshida, covering latest trends, challenges, and future developments in automotive technology.
Real-Time Pipeline Monitoring for the Energy Sector
Explore real-time pipeline monitoring for energy, covering data ingestion, architecture, WebAssembly, security, and analytics. Learn innovative solutions for edge-to-cloud monitoring and virtualized testing.
Fueling Event-Driven Designs: Unlocking Data Power with Open-Source Technologies
Explore real-time data capture using Debezium, Apicurio Registry, and Strimzi. Learn to stream database changes to Kafka, visualize with ksqldb and kafka-ui, and harness event-driven architectures.