Get in Control of Your Workflows with Airflow
Discover how to manage complex workflows using Airflow in this 26-minute conference talk from EuroPython 2016. Explore the core concepts of Airflow, an open-source Python package developed by Airbnb, and learn to define custom workflows using Python code. Gain insights into extending Airflow's functionality through task operators and UI blueprints, and hear real-world experiences from Blue Yonder's implementation. Master the art of creating directed acyclic graphs for workflow visualization, utilize out-of-the-box task operators, and develop custom tasks for specific needs. Understand how Airflow's Python-based configuration empowers users to efficiently control and monitor data processing workflows, addressing common challenges such as scheduling, dependency management, and error tracking.