Главная
Study mode:
on
1
Introduction
2
Our Data Platform
3
Our ETL Framework
4
Schema Revolution
5
Git Integration
6
Benefits of Delta Lake
7
Deployment
Description:
Explore the development of Mars Petcare's cloud-based Data Lake solution, the Petcare Data Platform, in this 27-minute conference talk. Learn how the Kinship Data & Analytics division leveraged Microsoft Azure, Delta Lake, and Databricks to create 'Kyte', a custom Spark ETL pipeline tool. Discover the advantages of migrating from Azure Data Factory to a Spark-heavy ETL design and Delta Lake-driven platform. Gain insights into using Delta Lake for ETL configurations and the creation of a bespoke UI for monitoring and scheduling Spark pipelines. Understand the benefits of this approach in supporting Mars Petcare's mission of making a better world for pets, including how Delta Lake is utilized to expose data to Data Scientists and the advantages of a Databricks & Spark ETL solution over Azure Data Factory.

Building the Petcare Data Platform with Delta Lake and Spark ETL Pipeline

Databricks
Add to list
0:00 / 0:00