Streamlined Monitoring of Databricks Workflows with Deloitte
Discover Deloitte's Automated Workflow Monitoring solution for Databricks, offering streamlined tracking of data engineering pipeline statuses with minimal overhead and a simplified view, built on Databricks' SDK and SQL dashboards.
Developing Data Pipelines with Branch Deployments - A New Approach
Explore a new approach to data pipeline development using branch deployments, enabling safer testing on production data and improving productivity, data reliability, and cost-efficiency.
Fabricator - Streamlining Declarative Feature Engineering at DoorDash
Découvrez comment DoorDash a développé Fabricator, un framework complet pour rationaliser l'ingénierie des caractéristiques déclaratives, orchestrant efficacement 1400 tâches quotidiennes et gérant 2,2 billions de valeurs.
Taking Good Flight with Reliable Data at American Airlines
American Airlines' data modernization journey: Enhancing efficiency, scalability, and reliability through cloud transition and data observability, leading to reduced downtime and faster insights for the world's largest airline.
Unlocking the Lakehouse with Efficient Data Pipelines - Capital One's Journey
Explore Capital One's data-driven transformation using Databricks Lakehouse, featuring efficient data pipelines, Delta Lake, and best practices for building a centralized data ecosystem.
Data Productivity at Scale - Improving Data Pipeline Development with SQLMesh
Explore SQLMesh, an open-source project revolutionizing data pipeline development with virtual environments, automatic contracts, and accurate change previews for improved efficiency and reliability.
Improve Your ML Projects - Embrace Reproducibility and Production Readiness with Kedro
Learn to enhance ML project efficiency using Kedro framework, focusing on reproducibility, flexibility, and production readiness. Gain hands-on Python experience for seamless integration.
Testing Data Pipelines - Techniques for Ensuring Data Flow Integrity
Explore effective techniques for testing data pipelines, ensuring smooth data flow and quick problem resolution. Learn about unit, integration, and end-to-end tests, plus unique methods like data snapshot testing.
Databricks Workflows: Practical How-Tos and Demos
Explore Databricks Workflows for efficient data pipeline orchestration. Learn best practices, serverless jobs, job repair, notifications, event-driven integrations, and effective triggers for optimized data intelligence.
Inside The Hybrid Data Strategy Powering Atlassian's Soaring Growth
Atlassian's hybrid data strategy combines Databricks and ActionIQ, enabling faster marketing execution, higher-converting audiences, and complex use cases like omnichannel journeys and personalization without sacrificing governance.
One Dataset and Three Use Cases with Microsoft Fabric
Leverage World Bank Open Data with Microsoft Fabric using various methods, model data, and create stunning reports in this demo-filled session on end-to-end solutions.
Petabyte-Scale Data Migrations to Google Cloud - Best Practices and Case Study
Migrate petabyte-scale data pipelines to Google Cloud, optimizing performance and reducing costs. Learn from Globo's experience in handling massive concurrent users for major events.
Real-Time Data Transformation and Analytics with dbt Labs
Explore real-time data transformation and analytics with dbt Labs. Learn about Modern Data Stack, streaming analytics, and best practices for building efficient data pipelines and democratizing data access.
Accelerate Insights with Natural Language-Driven Analytics in BigQuery
Explore BigQuery's new natural language-driven analytics experience for faster data-to-insights journey across discovery, processing, querying, and visualization.
How to Define a Storage Infrastructure for AI and Analytical Workloads
Optimize AI/ML data pipelines at scale by selecting the right storage solutions for data preparation, training, inference, and serving. Integrate with Compute Engine, GKE, or Vertex workflows.
Making Big Data Useful Data
Explore strategies for transforming big data into actionable insights, focusing on user-friendly tools and frameworks to empower business users in data analysis and decision-making.
Keep Your Cache Always Fresh with Debezium
Explore how to maintain synchronized read views in distributed caches using change data capture, Debezium, Apache Kafka, and Infinispan. Learn to create denormalized data views and implement cross-site replication for cache updates.
Using Open Source Tech to Swap Out Components of Your Data Pipeline
Explore open-source technologies for flexible data pipelines, from Apache Kafka to BigQuery, with easy component swapping and cloud deployment options.
Reducing Complexity and Increasing Performance with Trino
Explore Trino's capabilities for efficient data querying across diverse sources, optimizing pipelines, and enhancing analytics performance in complex data environments.
Using Kafka at Scale - A Case Study of Microservices Data Pipelines at Evernorth Health Services
Explore a case study on implementing Kafka-based microservices data pipelines at scale in Evernorth Health Services, showcasing real-world applications and challenges.
Deeply Declarative Data Pipelines with Flink and Kubernetes
Explore declarative streaming data pipelines on Kubernetes using Flink and SQL. Learn to simplify development, integrate systems, and optimize configurations for efficient stream processing.