Automating Data Pipelines with Python and GitHub Actions - Code Walkthrough
Automate data pipelines using Python and GitHub Actions. Learn to create ETL scripts, set up repositories, configure workflows, and implement ML applications for efficient data processing.
Accelerating Big Data - Modern Trends, Enable Product Analytics
Explore modern trends in big data acceleration and enable effective product analytics for improved decision-making and business insights.
Cloud Agnostic Data Platform with DBT, Trino, Iceberg and MinIO
Explore a hands-on, cloud-agnostic data pipeline using TrinoDB, DBT, MinIO, and Apache Iceberg for scalable and lightweight analytics, suitable for both cloud and on-premise solutions.
DBT and Python - How to Write Reusable and Testable Pipelines
Develop reusable and testable data pipelines using DBT and Python. Learn best practices for version control, DAGs, and unit testing in SQL and Python-based models.
From Pandas to Production: ELT with dlt
Explore data load tool (dlt) to streamline data science workflows, overcome roadblocks, and transition smoothly from exploration to production. Learn best practices for data handling and managing failures with real-life examples.
Simplifying Machine Learning Lifecycle Management in Healthcare
Explore end-to-end ML lifecycle in healthcare using Azure services. Learn data ingestion, model creation, deployment, and visualization for real-time patient predictions and batch scoring.
MLOps Without Much Ops - Building Efficient Machine Learning Systems
Discover modern, no-nonsense data pipelines for efficient machine learning systems. Learn PaaS advantages and explore real-world applications with open-source code. Gain insights on ML's future for organizations of all sizes.
Retail Transactions at Scale for AI - From PoC to Production
Discover how to scale AI solutions for retail transactions, from handling millions of unstructured data points to enabling enterprise-wide AI applications and reporting capabilities.
Version Control for Lakehouse Architecture - Essential Practices and Benefits
Discover how to implement data version control using lakeFS for improved data quality, reproducibility, and experimentation in Databricks data/ML pipelines, enhancing overall product value.
Ocean Cleanup's Efforts to Remove Ocean Plastic - Leveraging Databricks
Databricks supports Ocean Cleanup's mission to remove 90% of ocean plastic by 2040, enhancing computer vision models, developing impact reporting, and streamlining data architecture for more efficient operations.
Building a Trusted Data Foundation for AI on Databricks
Modernize data engineering for AI with Qlik's integration of Databricks AI Functions, automating real-time data pipelines to enhance AI-driven analysis and ML model development on the Databricks platform.
Delta Live Tables in Depth - Best Practices for Intelligent Data Pipelines
Explore best practices, new features, and future developments for Delta Live Tables with Databricks experts. Learn pipeline development, data quality, and serverless compute for intelligent data pipelines.
Data Pipelines Evolution from Batch to Streaming
Explore the evolution from batch to streaming data pipelines using Apache Flink and Kafka. Learn strategies for seamless transition, including query-based connectors, change data capture, and handling late events.
Streaming vs. Batch Processing: The Future of Data Architecture
Explore the future of data processing with industry experts as they debate streaming vs. batch, adoption challenges, and best practices for real-time data architecture.
The Metadata-Driven Data Warehouse
Streamline data warehouse development using metadata-driven approaches. Learn efficient techniques for Insert/Update/Delete operations and pipeline management.
Introduction to Databricks Delta Live Tables
Accelerate data pipeline development with Databricks Delta Live Tables. Learn setup, ingestion, table creation, pipeline execution, and data validation for faster, more reliable projects.
Navigating the Skies of Data: A Flight Through Feature Stores
Explore Feature Stores' role in streamlining ML projects, acting as a control tower for efficient feature management in data processes.
Building Stream Processing Applications with Ruby and Meroxa - RubyConf 2022
Explore real-time stream processing using Ruby and the Turbine framework. Learn to build and deploy sophisticated applications without complex tooling or unfamiliar paradigms.
Orchestrating Scalable Data Pipelines with Apache Toree, YuniKorn, Spark, and Airflow
Explore integrated use of Apache Toree, YuniKorn, Spark, and Airflow for efficient, scalable data pipelines. Learn to leverage these projects for optimized data processing and workflow orchestration.
Declarative ETL Pipelines with Delta Live Tables - Modern Software Engineering for Data Analysts and Engineers
Modernize ETL pipelines using Delta Live Tables for efficient data management. Learn development, production, and best practices for reliable, testable, and scalable data processing.
Machine Learning in Astrophysics and Cosmology
Explore how astrophysics and cosmology leverage machine learning to analyze vast datasets, pioneering new methods and infrastructure for handling big data in scientific research.