A New Data Model for Time Series Data Processing
Innovative time series data model enhancing ingestion, query performance, and storage efficiency. Introduces "one table per collection point" and "supertable" concepts, with benchmark results demonstrating effectiveness.
Leveraging Data in Motion in a Cloud-First World - Jun Rao
Explore Apache Kafka's journey, its cloud-native potential, and real-world applications. Learn how Fortune 100 companies leverage this platform for data-in-motion, revolutionizing data engineering and analytics workflows.
Streaming Graphs - Real-Time Event Processing with Quine
Explore Quine, an open-source streaming graph solution for efficient event stream interpretation, offering real-time query answers and scalability to millions of events per second.
Airflow: Save Money Using Deferrable Operators - Introduction and Implementation
Learn to optimize Apache Airflow tasks using deferrable operators, saving resources and costs. Explore implementation, advantages, and custom development for efficient workflow management.
Fast, Cheap and Easy Data Ingestion with AWS Lambda and Delta Lake
Explore event-driven data ingestion using AWS Lambda with Delta Lake. Learn to process, update, and query Delta tables efficiently in Python and Rust for a more responsive data platform.
Data Engineering with Rust and Delta Lake
Explore data engineering with Rust, leveraging Delta Lake, data fusion, and arrow for high-performance pipelines. Learn to ingest, enrich, manage, and export data using Rust's powerful capabilities.
Safe, Concise, Performant Code with the Scala Programming Language
Discover Scala's unique blend of conciseness and performance. This talk explores how Scala combines elegance with scalability, from simple REPL one-liners to complex real-world applications.
Best Practices for Unit Testing PySpark
Learn effective techniques for unit testing PySpark code, including local and CI testing, code structuring for testability, and integration testing with clusters for enhanced reliability and easier refactoring.
Exploring UDTFs and Polymorphism in PySpark
Learn to create and enhance User-Defined Table Functions in PySpark, including polymorphic UDTFs for flexible data processing. Gain practical insights through examples and expert guidance.
Quarkus Meets AI - Build Your Own LLM-Powered Application
Explore building AI applications with LLMs using Quarkus and Langchain4j. Learn data ingestion strategies and integration with Apache Camel for comprehensive resource utilization.
Apache Kafka and the Rise of the Streaming Platform
Explore streaming platforms with Apache Kafka as the core, revolutionizing data flow management. Learn how this technology acts as a central nervous system for modern digital businesses.
Quarkus Meets AI - Build Your Own LLM-Powered Application
Explore building AI applications with LLMs using Quarkus and Langchain4j. Learn data ingestion strategies and integration with Apache Camel for comprehensive resource utilization in Java-based AI development.
From Data Engineering to MLOps: Building a Scalable ML Platform - Reverb's Evolution
Discover how Reverb transformed their data engineering team into MLOps, exploring scalable ML platform development, developer experience optimization, and effective monitoring strategies using Ray and Anyscale.
Building AI Pipelines with Hugging Face and Kedro - From ChatGPT to Production
Discover how to build robust AI systems using Hugging Face's open models and Kedro pipelines, moving beyond commercial AI to create customizable and transparent machine learning solutions.
Data Contracts Implementation with DataHub - A Practical Guide
Master practical implementation of data contracts using DataHub's metadata platform to enhance data quality management and create a harmonized control plane across fragmented systems.
Data as Code - Project Nessie and Git-like Experience for Apache Iceberg Tables
Discover how Project Nessie enables multi-table transactions and Git-like workflows for Apache Iceberg tables, revolutionizing data lakehouse management and engineering practices.
CICD Pipelines for dbt - Choosing Between DIY and Cloud Solutions
Explore the trade-offs between building custom CICD pipelines for dbt projects versus using dbt Cloud's built-in solutions, focusing on Slim CI and documentation deployment strategies.
Rise, Fall and Re-Rise of the Semantic Layer - Evolution and Future of Data Management
Explore the evolution of semantic layers in data management, from traditional systems to modern metric stores, while understanding key challenges and future directions in data truth management.
Data as the Programming Interface - The Future of AI Development
Explore how data-centric approaches revolutionize AI development, focusing on specialized evaluation, fine-grained analysis, and scalable solutions for capturing human feedback in enterprise systems.
Leveraging Large Language Models for ETL, Analytics, and Modern Data Stack
Discover how Large Language Models revolutionize data workflows, from ETL processes to analytics automation, with practical insights on implementation challenges and production deployment strategies.
From AI Conversations to Data-Driven Decisions - Unlocking the Future of Customer Engagement
Explore how AI assistant Tobi leverages machine learning, data engineering, and analytics to predict customer needs and deliver personalized offers through integrated backend systems.