Tutorial de Apache Airflow: Desde Puesta en Marcha hasta API - Creación de DAGs y Uso de CLI
Aprende a instalar Apache Airflow, crear DAGs, utilizar operadores, ejecutar workflows, comunicar tareas y manejar el cliente CLI y API REST en este tutorial práctico.
Orchestration of Data Pipelines Using Airflow
Learn to orchestrate data pipelines with Airflow, integrating various operators like Bash, Python, Spark, Email, and Slack for efficient workflow management and automation.
Data Governance with Airflow and ECS Anywhere
Explore data governance compliance using Airflow and AWS ECS Anywhere for EU companies processing US data, ensuring legal adherence while maintaining efficient ETL processes.
Airflow's Limitations for MLOps - Challenges and Alternatives
Explore challenges in MLOps, data platforms, and Airflow with Stephen Bailey. Gain insights on modular job design, data classification, and practical approaches to data engineering and machine learning workflows.
GenAI Pipeline for Content Generation with Apache Airflow
Explore Apache Airflow's key features for building GenAI pipelines, including TaskFlow API, automatic retries, branching, and dynamic task mapping. Learn to overcome challenges in content generation workflows.
Realtime Data Streaming - End-to-End Data Engineering Project
Build a real-time data streaming pipeline using Apache Airflow, Kafka, Spark, and Cassandra. Learn end-to-end data engineering, from ingestion to processing and storage, all containerized with Docker.
Football Data Analytics: Azure End-to-End Data Engineering Project
Extract, clean, and analyze football data using Apache Airflow, Azure services, and Tableau. Build a comprehensive data pipeline from Wikipedia to interactive visualizations.
Football Data Analytics: Azure End-to-End Data Engineering Project - Part 2
Extract, store, and analyze football data using Azure services and Apache Airflow. Develop end-to-end data engineering skills through hands-on implementation of a real-world project.
Reddit Data Pipeline Engineering with AWS - End-to-End Data Engineering
Build a comprehensive Reddit data pipeline using AWS services, integrating Airflow, S3, Glue, Athena, and Redshift for efficient ETL processes and analytics.
Apache Airflow with Spark for Data Engineers - Full Course
Build end-to-end data engineering projects using Apache Airflow, Docker, Spark, and multiple programming languages. Learn to create and submit jobs to Spark clusters for processing and view real-time results.
Treating Prompt Engineering More Like Code - Promptimize and Test-Driven Development for AI
Explore scientific prompt evaluation with Promptimize. Learn about open-sourcing, test suites, and AI challenges. Gain insights on prompt optimization, deterministic evaluation, and the evolving landscape of AI prompt engineering.