Orchestrating Python Workflows in Apache Airflow

Orchestrating Python Workflows in Apache Airflow

Dive into the world of workflow orchestration with this 26-minute conference talk from EuroPython 2023. Explore Apache Airflow, an open-source Python library for automating complex code and integrating with various data sources. Learn about its strengths, weaknesses, recommended features, and potential pitfalls based on five years of production experience. Discover how to create custom operators in Airflow and gain insights into the technology powering data pipelines for major companies like Airbnb, Slack, and Walmart. Understand the integrated UI and API capabilities that cater to both human and programmatic needs in this comprehensive overview of Apache Airflow's workflow orchestration capabilities.