[SK TECH SUMMIT 2022] EMR과 Airflow로 구축하는 Mobility Data Platform
Description:
Learn how to migrate and build a Mobility Data Platform using AWS services in this technical conference talk from SK TECH SUMMIT 2022. Discover the journey of transitioning from an on-premise system to AWS, focusing on the integration of EMR and Airflow. Explore the process of moving large-scale data to AWS S3 using DataSync, setting up Hadoop clusters with EMR, and implementing ETL scheduling and data pipeline execution with Airflow. Gain insights into automated ETL workflow migration strategies, infrastructure as code implementation using Terraform, and cost optimization techniques for EMR. The presentation, delivered by Seunghoon Baek from TMap Mobility, draws from extensive experience in search development at SK Communications, Kakao Shopping, and TMap services, offering practical solutions for AWS service performance optimization and cost efficiency in mobility data platforms.
Building Mobility Data Platform with EMR and Airflow