Genomic-scale Data Pipelines - Building Serverless Bioinformatics Cloud Solutions
Explore genomic-scale data pipelines with cloud architects Lynn Langit and Denis Bauer. Learn about serverless bioinformatics, MedTech solutions, and cutting-edge approaches to handling large-scale genomic data.
Unlocking Financial Data with Real-Time Pipelines Using Apache Technologies
Discover how to build powerful real-time data pipelines using Apache Flink, NiFi, Kafka, and Iceberg for processing financial data efficiently, enabling swift decision-making and regulatory compliance.
Apache Airflow 2.x - Modern Workflow Orchestration Features and Improvements
Discover the modern evolution of Apache Airflow 2.x, exploring new UI features, orchestration methods, and testing capabilities while maintaining backward compatibility for workflow management.
Introduction to Orchestrating ETL Processes with Apache Airflow
Master Apache Airflow's workflow management capabilities for ETL processes through practical examples. Learn core concepts like DAGs, Tasks, Variables, and XComs while building efficient data engineering pipelines.
Simplifying Streaming Data Transforms With WASM
Explore WASM-based data transformation for simplified, high-performance real-time applications and data pipelines. Learn architecture, use cases, and see a live demo with Redpanda.
Design Cloud to Edge Architecture Patterns with Azure Arc
Explore Azure Arc for deploying and managing applications across cloud and edge environments, with demos on Kubernetes deployment, AI-enhanced operations, and cross-platform infrastructure management.
Data Pipeline - Python Web Scraping for Beginners
Learn to create custom data pipelines for efficient web scraping in Python, streamlining data collection and processing for beginners.
Streaming Graphs - Real-Time Event Processing with Quine
Explore Quine, an open-source streaming graph solution for efficient event stream interpretation, offering real-time query answers and scalability to millions of events per second.
CICD Pipelines for dbt - Choosing Between DIY and Cloud Solutions
Explore the trade-offs between building custom CICD pipelines for dbt projects versus using dbt Cloud's built-in solutions, focusing on Slim CI and documentation deployment strategies.
Building an Integrated Data Asset Platform with Apache DolphinScheduler - DataOps Track
Discover how Apache DolphinScheduler powers Unicom Digital Technology's integrated data asset platform, exploring key features and implementation strategies for efficient data operations.
How Replit Trained Their Own LLMs - LLM Bootcamp
Comprehensive guide to training custom LLMs: from data processing and tokenization to evaluation and deployment, with insights on tools, best practices, and essential skills for LLM engineers.
Leveraging OpenTelemetry for Your Prometheus Pipeline
Explore OpenTelemetry's integration with Prometheus, covering receivers, exporters, migration paths, and common scenarios for leveraging OTel in your metrics pipeline.
Global School Connectivity DataOps Platform
Explore the development of an open-source DataOps stack managing connectivity data for millions of schools, covering ingestion, pipelines, quality, and analytics integration.
Solving Migration Pipelines
Explore strategies for solving migration pipeline challenges in cloud-native environments, focusing on efficiency and scalability.
DevOps Best Practices for DataOps Mesh
Discover DevOps best practices for implementing DataOps Mesh, enhancing data management and collaboration in complex environments.
Build a Deep Facial Recognition App - Preparing Data for Deep Learning - TF Dataloader
Learn to build a deep facial recognition app using TensorFlow, covering data preparation, image processing, and pipeline creation for training and testing a Siamese Neural Network model.
Democratizing Data Quality Through a Centralized Platform at Zillow
Explore Zillow's centralized platform for data quality, enabling stakeholders to define expectations, perform validations, and monitor data health across complex organizations.
Deep Learning for Real Time Streaming Data with Kafka and TensorFlow - ODSC East 2019
Learn to process real-time streaming data using Apache Kafka and TensorFlow's KafkaDataset module for efficient machine learning applications in mission-critical scenarios.
Stream All Things - Patterns of Modern Data Integration
Explore patterns for modern data integration using Apache Kafka. Learn to build fast, scalable, and manageable data pipelines for reliable stream processing and low-latency applications.
Building a Batch Processing Platform for Data Pipelines Using Argo and Kubernetes
Explore Intuit's scalable batch processing platform using Kubernetes and Argo for efficient data pipeline management, addressing scheduling, orchestration, and dependency challenges in AI and data engineering.
Build or Buy: Observability Data Pipelines 101
Explore efficient observability data pipeline management for cloud services. Learn to handle massive data volumes, reduce costs, and optimize processing across teams and tools.