Orchestrating Data Assets Instead of Tasks, With Dagster - Sandy Ryza
Explore data orchestration with Dagster, focusing on managing data assets over tasks. Learn about pipeline development, testing, deployment, and monitoring in data engineering and machine learning.
Making Data the Central Feature: From Task-Oriented to Data-Oriented Workflows
Explore the shift from task-oriented to data-oriented workflows in modern data engineering, focusing on tools and approaches that prioritize data assets over process management.
Developing Data Pipelines with Branch Deployments - A New Approach
Explore a new approach to data pipeline development using branch deployments, enabling safer testing on production data and improving productivity, data reliability, and cost-efficiency.
Demystifying Apache Airflow - Separating Facts from Fiction
Unpack critiques of Apache Airflow, analyze its strengths and weaknesses, debunk myths, and compare with competitors. Make informed decisions about workflow management platforms.
Declarative Orchestration: Why You've Been Thinking About the Wrong DAG the Entire Time
Explore declarative, asset-oriented orchestration in data pipelines. Learn how this approach simplifies tasks, ensures data integrity, and implements Data Mesh as a living asset graph.
Rising Tides with Radical Transparency - Open Sourcing Your Data Platform
Explore the journey of open-sourcing a data platform, including challenges, cultural shifts, and innovative implementations. Learn to leverage open source for enhancing projects and contributing to the data community.