Columnar Data Formats Enabling Serverless Data Analysis at Scale
Explore the power of columnar data formats and serverless computing for efficient data analysis at scale in this 52-minute conference talk from Devoxx. Delve into the benefits of Parquet and ORC formats for optimizing query performance and costs in analytics scenarios. Learn how combining columnar storage with serverless platforms like AWS Lambda can simplify big data analytics, data collection, and ETL orchestration while reducing total ownership costs. Discover strategies for addressing data challenges, implementing effective database solutions, and leveraging columnar data layouts. Gain insights into use cases for traditional data warehouses and data lakes, and explore components such as data catalogs, classification techniques, and partitioning strategies. Understand the impact of these technologies on long queries and big data processing, and learn how to harness dark data for valuable insights.