DataDevOps for the Modern Data Warehouse on Microsoft Azure
Explore DevOps principles applied to Big Data Analytical Pipelines in this comprehensive conference talk. Dive into the Modern Data Warehouse architecture, designed to address the challenges of Big Data, Machine Learning, and Advanced Analytics. Learn how to build CI/CD pipelines for multi-source data warehouses using Microsoft Azure Data Platform technologies, including Data Factory, Databricks, Data Lake Gen2, and AzureDevOps. Through extensive demonstrations, gain insights into validating data, ensuring data quality, and implementing DevOps practices in data pipelines. Discover how to leverage Azure Key Vault, create release pipelines, manage environments, and utilize PowerShell scripts for efficient data operations. Explore monitoring techniques using Application Insights and gain a deeper understanding of the Modern Data Warehouse ecosystem.