Power BI and Azure Synapse Analytics: Beginning Your Journey
Explore Power BI and Azure Synapse Analytics integration, learning effective data leveraging techniques to maximize insights and optimize your data strategy.
The Future Roadmap for the Composable Data Stack
Explore advancements in data processing with Wes McKinney. Learn about Parquet, Arrow, and interface libraries like Ibis and tidyverse. Gain insights into current project states and future innovations.
Streaming CDC Data from PostgreSQL to Snowflake - Challenges and Solutions
Uncover streaming ETL secrets: Postgres to Snowflake data ingestion, change data capture, dynamic tables, and Snowpipe streaming. Learn solutions for lower latency, cost savings, and enhanced scalability.
Building a Flexible Data Platform for LLM Training Data
Explore LLM training data platforms: architecture, ingestion, preprocessing, and scaling challenges. Learn from Cohere's experience in managing petabyte-scale datasets for high-quality model training.
Evolving Data Pipelines at Scale with SQLMesh
Discover SQLMesh: revolutionize data pipeline development with virtual environments, automated testing, and accurate change previews inspired by industry giants.
Open Data Foundations Across Hudi, Iceberg and Delta - OneTable Integration
Explore OneTable's innovative approach to optimizing data across Hudi, Iceberg, and Delta Lake. Learn how it enhances workloads for various query engines and streamlines data analysis.
Why Streaming SQL? The Semantics and Challenges of Applying SQL to Unbounded Data
Explore SQL's application to unbounded data streams, covering time-oriented elements, continuous pipeline challenges, and insights for data professionals.
Building Real-Time Customer-Facing Analytics at Scale - From Twilio to Propel
Explore real-time customer analytics architecture at Twilio, including benefits, challenges, and sub-second latency impact. Learn how these insights influenced Propel's approach to diverse analytics use cases.
Design Patterns for Data Pipelines Beyond ETL
Discover essential design patterns for managing complex data pipelines in Python, focusing on advanced transformation techniques and Django integration for robust ETL workflows.
Hyper Speed! When Big Data Blooms
Explore techniques for processing massive data volumes at high speeds, with insights from Report URI's handling of 65,000 JSON payloads per second in global telemetry.
Building a Telegram Bot with Apache Kafka and ksqlDB
Hands-on demonstration of building a Telegram bot using Apache Kafka and ksqlDB, showcasing real-time data processing, querying, and integration capabilities without additional datastores.
Tips and Tricks for Robust Big Data Applications
Discover essential tips for optimizing big data applications, including storage efficiency, data quality assurance, governance, and automation tools for improved performance and compliance.
Continuous Delivery for Machine Learning - Patterns and Pains
Explore challenges and patterns in implementing Continuous Delivery for data-driven systems with machine learning, focusing on pain points and practical solutions for complex architectures.
Product Analytics and Personalization for Facebook and Instagram Shops
Discover how Meta leverages data analytics for product development through a case study of Facebook and Instagram Shops, exploring the crucial roles of Data Engineering and Data Science.
Building Reproducible ML Processes with an Open Source Stack
Discover how to build truly reproducible machine learning experiments by integrating MLFlow, Git, lakeFS, and infrastructure-as-code for consistent data, code, and environment management.
Real-Time Event Streaming Patterns for AI-Native Applications
Explore how event streaming patterns power AI applications through real-time analytics, data enrichment, and stream processing for enhanced intelligence and reliable infrastructure.
Data Contracts Are Good for AI
Discover how data contracts and the ODCS standard enhance AI development, focusing on Bitol's open-source tools for modern data engineering and contract enforcement.
Exploring Cribl Stream and Cribl Edge - Data Collection and Observability Pipeline Demo
Discover how to manage observability pipelines with Cribl Stream and Edge, enabling flexible data collection, reduction, enrichment, and routing across your infrastructure for optimized business operations.
Solving Security's Data Problem with Cribl Stream - Building Sustainable Security Architecture
Master data pipeline optimization techniques to enhance security operations, reduce noise, and build sustainable architectures for effective threat detection and response.
Optimizing Telemetry Data Analysis and Log Management with Mezmo
Master telemetry data optimization techniques for DevOps and SREs - from data profiling and cost management to real-time monitoring and sensitive data handling in observability platforms.
Qlik Talend Cloud - AI-Assisted Data Engineering Overview and Demo
Discover how Qlik Talend Cloud revolutionizes data engineering with AI-assisted automation, no-code options, and advanced data integration capabilities for building robust, trustworthy data pipelines.