Главная
Study mode:
on
1
Introduction
2
System Architecture
3
Setting up the project
4
Creating Dimensional Modelling with Apache Airflow
5
Creating Apache Airflow Hook for Kafka
6
Creating Apache Airflow Hook for Apache Pinot
7
Connecting Apache Pinot to Kafka
8
Batch Data Ingestion for Apache Pinot
9
Setting up Apache Superset for Data Visualisation
10
Creating Superset Dataset for Visualisation
11
Creating Apache Superset Realtime DW Dashboard
12
Wrapping up
13
Outro
Description:
Embark on a comprehensive journey to build a real-time data warehouse from scratch in this 2-hour video tutorial. Design and implement a complex real-time data warehouse architecture, set up Apache Airflow, Kafka, and Apache Pinot for seamless data pipelines, and develop custom Apache Airflow hooks for Kafka & Pinot integration. Learn to ingest batch and streaming data into Apache Pinot for real-time analytics, create a dynamic dashboard with Apache Superset to visualize evolving data in real-time, and apply dimensional modeling for better data organization and reporting. Follow along as the instructor guides you through each step, from system architecture and project setup to creating dimensional models, connecting various components, and finally setting up a real-time dashboard. Gain practical experience in big data engineering, ETL processes, and data analytics while building a complete end-to-end data engineering project.

Building Realtime Data Warehouses from Scratch - End-to-End Data Engineering Project

CodeWithYu
Add to list
0:00 / 0:00