Главная
Study mode:
on
1
Introduction
2
Agenda
3
Traditional Data Warehouse
4
Data Lake
5
Data Lake tiers
6
Validate
7
Ensure
8
Demo
9
Common Question
10
DevOps
11
Data Pipelines
12
Data Bricks
13
Sequel Data Warehouse
14
Data Factory
15
Data Factory Workflow
16
Key Vault
17
Sample Release
18
Commit to Master
19
Static artifacts
20
Seco packages
21
Creating a release
22
Release Pipeline
23
Staging Environment
24
Variables
25
Release
26
Override parameters
27
PowerShell script
28
Backpack task
29
Monitoring
30
Monitoring Metrics
31
Application Insights
32
Summary
Description:
Explore DevOps principles applied to Big Data Analytical Pipelines in this comprehensive conference talk. Dive into the Modern Data Warehouse architecture, designed to address the challenges of Big Data, Machine Learning, and Advanced Analytics. Learn how to build CI/CD pipelines for multi-source data warehouses using Microsoft Azure Data Platform technologies, including Data Factory, Databricks, Data Lake Gen2, and AzureDevOps. Through extensive demonstrations, gain insights into validating data, ensuring data quality, and implementing DevOps practices in data pipelines. Discover how to leverage Azure Key Vault, create release pipelines, manage environments, and utilize PowerShell scripts for efficient data operations. Explore monitoring techniques using Application Insights and gain a deeper understanding of the Modern Data Warehouse ecosystem.

DataDevOps for the Modern Data Warehouse on Microsoft Azure

NDC Conferences
Add to list
00:00
-00:34