Apache Arrow DataFusion: Vectorized Execution Framework for Maximum Performance

Apache Arrow DataFusion: Vectorized Execution Framework for Maximum Performance

Explore the powerful Apache Arrow DataFusion framework in this 28-minute conference talk by Liu Kun, an eBay big data engineer and Apache Arrow PMC member. Dive into the fast, extensible, and vectorized execution framework that leverages Arrow as its in-memory data format and is implemented in Rust. Discover DataFusion's architecture, extension capabilities, and use cases. Learn how to integrate DataFusion into database or query system implementations, taking advantage of its extreme performance while avoiding the need to recreate a query engine. Gain insights into DataFusion's history, extension interfaces (including UDFs, logical plans, and execution plans/nodes), and current applications in various scenarios.