Apache Arrow and Substrait - The Secret Foundations of Data Engineering
Explore Apache Arrow and Substrait's impact on data engineering, including their role in interoperability, query planning, and building efficient database systems with minimal code.
Scaling to Millions of ML Models to Solve the Problems of SRE and Security
Explore scaling ML models for SRE and security, using open-source technologies to improve zero trust frameworks and infrastructure diagnosis on Kubernetes and service mesh environments.
Better Together - Unleashing the Synergy of Pandas, Polars, and Apache Arrow
Discover how to optimize data engineering workflows by combining Pandas and Polars with Apache Arrow, learning techniques to build faster, more efficient data pipelines for scalable solutions.
Forecasting Time Series with Arrow and WASM
Explore novel time-series processing pipeline using Apache Arrow, S3, Polars/DuckDB, and WASM for efficient forecasting and MLOps deployment.
The Future Roadmap for the Composable Data Stack
Explore advancements in data processing with Wes McKinney. Learn about Parquet, Arrow, and interface libraries like Ibis and tidyverse. Gain insights into current project states and future innovations.
Implementing InfluxDB IOx Using Apache Arrow, DataFusion, and Rust
Discover how modern database systems are built using Apache Arrow, DataFusion, and Rust, exploring the architecture and implementation of InfluxDB IOx through deconstructed database approaches and open-source components.
Time Series Analytics with Apache Arrow, Pandas and Parquet - A Beginner's Guide
Discover how to build high-performance time series analytics applications using Apache Arrow Flight, Parquet, and Pandas, focusing on efficient data processing and querying techniques for columnar databases.
Maximizing Real-Time Data Processing with Apache Kafka and InfluxDB - A Comprehensive Guide
Learn to create powerful data pipelines by combining Apache Kafka and InfluxDB for real-time processing, analysis, and visualization of large-scale data, especially in IoT applications.
Python and SQL: Better Together, Powered by DuckDB
Explore DuckDB's integration with Python for efficient data processing, combining SQL and dataframe capabilities to handle large datasets with ease and speed.
Making Moves with Arrow Data: Introducing Arrow Database Connectivity (ADBC)
Explore Arrow Database Connectivity (ADBC), a new Apache Arrow subproject for Arrow-based database access. Learn its goals, use cases, and communication with data APIs using Arrow-native in-memory data.
Ten Years of Building Open Source Standards: From Parquet to Arrow to OpenLineage
Explore the journey of building open-source data standards, from Apache Parquet to Arrow to OpenLineage. Gain insights into successful project ideation, growth, and ecosystem impact in data management.
Time Series Analytics with Apache Arrow, Pandas, and Parquet - A 101 Introduction
Explore efficient time-series data handling using Apache Arrow and Parquet. Master open-source technologies for high-performance analytics applications.
Building an Open Data Service Layer with Arrow, DuckDB and Semantic Layer
Explore FlexQuery, an open-source solution revolutionizing data services with Arrow, DuckDB, and semantic layers. Learn to create high-performance, scalable analytics environments using Python and cutting-edge technologies.
Ten Years of Building Open Source Standards in Data Engineering
Explore a decade of open-source standards development, from Parquet to Arrow. Gain insights into community-driven innovation and the evolving landscape of data engineering.
Building a Unified Feature Platform with DuckDB and Arrow
Explore ML feature platform development using DuckDB and Arrow. Learn scaling strategies, DuckDB extension implementation, DeltaLake interoperability, and performance comparison with Spark.
Building InfluxDB 3.0 with Apache Arrow, DataFusion, Flight and Parquet
Explore InfluxDB 3.0's innovative design using Apache Arrow, DataFusion, Flight, and Parquet. Discover how this deconstructed database system enables fast, high-quality data analysis and advanced interoperability.
Open Standards Make MLOps Easier and Silos Harder - MLOps Podcast Episode 234
Explore open standards in MLOps for easier data system integration, scalability, and collaboration. Learn about Ibis, Apache Arrow, and their impact on data engineering and machine learning workflows.
Neuralake - Complex Data, Simple Systems
Explore Neuralake: a multimodal data lake for managing complex neural data. Learn about its Rust-based stack, real-time ingestion, and scalable architecture for efficient data access and retrieval.
Python and SQL: Better Together, Powered by DuckDB
Explore DuckDB: A powerful database system integrating seamlessly with Python, offering efficient data processing and easy-to-use SQL capabilities for data scientists and analysts.
Open Standards for Open Lakehouses - Understanding Apache Iceberg, Parquet, Arrow, and Nessie
Discover how open-source standards like Apache Iceberg and Nessie power data lakehouses, enabling flexible and cost-effective data platforms while minimizing vendor lock-in and data movement challenges.
Integrating Apache Arrow on WASM Using Component Model
Discover how to integrate Apache Arrow with WASM Component Model, exploring efficient data handling techniques and practical applications across IoT, robotics, and cloud infrastructure.