Combining Machine Learning and MLflow with Lakehouse Architecture - Data Brew Episode 5
Explore how Quby leverages ML and MLflow with their lakehouse to extract value from IoT sensor data, moving from batch to real-time streaming for energy optimization and customer insights.
Data Mesh in Practice - Beyond the Data Lake for Fashion E-Commerce
Explore Zalando's journey from centralized Data Lake to distributed Data Mesh architecture, emphasizing data ownership, quality, and accessibility through decentralized domain-focused approach and Data Products concept.
Taking Machine Learning to Production with MLflow - New Features and Best Practices
Explore MLflow's latest features for productionizing machine learning, including model management, CI/CD, data schemas, and integration with PyTorch for seamless deployment and operation of ML applications.
Monitoring ML Models, IoT Data, and Quality Checks on Delta Lake - Quby's Approach
Discover how Quby monitors IoT data, ML models, and data quality using Databricks. Learn about Delta Lake, MLflow, and Slack integration for efficient data processing and model performance tracking.
Identifying Sensitive Data and Mitigating Risk in Apache Spark and Databricks
Discover sensitive data in Apache Spark and Databricks using BigID's platform. Learn to scale discovery, label data, and apply necessary guardrails for compliance and risk mitigation in data analytics.
How Delta Lake with Azure Databricks Can Accelerate Your Big Data Workloads by 100x
Explore advanced Delta Lake features on Azure Databricks for reliable, scalable data lakes. Learn optimizations for analytics and ML workloads, including file compaction, Z-Order partitioning, and unified batch/streaming.
Lakehouse Architecture with Delta Lake and Databricks
Dive into Lakehouse architecture fundamentals, exploring Delta Lake format and Databricks platform implementation for modern data management and analytics solutions.
Fixing Small Files Performance Issues in Apache Spark Using DataFlint
Master Apache Spark's file handling optimization techniques and learn to solve small files performance issues using DataFlint. Explore best practices for storage formats like Delta Lake and Iceberg.
Whitefox: Simple Table Format Sharing for Open Data Systems
Simplify data sharing with Whitefox, an open-source protocol for streamlined access management in table-format architectures, offering lightweight and native accessibility for Iceberg and Delta Lake.
Govern Your Self-Service Integration Processes
Implement clear governance for self-service data integration across diverse environments. Control and audit data loading, processing, and uploading with WOODY.IO.
Towards Multi-Table Transactions in Delta Lake
Explore managed-commits in Delta Lake: a new protocol enabling multi-table transactions, reliable commit semantics, and enhanced data governance for cloud storage-based data lakes.
Delta Lake for Polyglot Data and Machine Learning Workloads
Explore Delta Lake's power for data streams, analysis, and ML workloads. Learn about data quality monitoring, efficient job management, and leveraging Delta for auto-ML models and Spark Structured Streaming.
Delta Tensor: Efficient Vector and Tensor Storage in Delta Lake
Efficient tensor storage solution for machine learning workflows, using chunking and sparse encoding to reduce IO costs and improve storage efficiency in cloud-native Lakehouse environments.
Efficient Near Real-Time Event Ingestion using DLT - Insights and Lessons
Explore Nextdoor's shift to near-real-time streaming with DLT, enabling quick event querying for analysis and monitoring while reducing costs. Learn about challenges, lessons, and optimization techniques.
Shaping the Future of Data in India's Premier Fashion E-Commerce Platform
Optimizing petabyte-scale clickstream data analysis for fashion e-commerce: Migration to Delta Lake enhances scalability, cost-efficiency, and performance in retail analytics.
Data Driven: The Journey to Bring Real-Time ERP Data to a Lakehouse
Michelin's journey to real-time ERP data integration in a lakehouse, leveraging tools like Kafka, Oracle Golden Gate, and Databricks for faster inventory updates and enhanced business analytics.
Delta Kernel - Anatomy of a Delta Lake Connector
Explore Delta Kernel project for simplified Delta Lake connector creation. Learn architectural decisions, engineering considerations, and community usage in this informative session.
Unlocking the Lakehouse with Efficient Data Pipelines - Capital One's Journey
Explore Capital One's data-driven transformation using Databricks Lakehouse, featuring efficient data pipelines, Delta Lake, and best practices for building a centralized data ecosystem.
Accelerating Adoption of Data Lake for Streaming and Machine Learning Use Cases
Discover tools and features for efficient Databricks adoption in ML and streaming, focusing on automation and operational efficiency to accelerate implementation of Delta and Spark compute workloads.
Processing a Trillion Rows Per Day with Delta Lake at Adobe
Scaling data processing at Adobe: managing thousands of Delta tables, handling terabytes daily, and powering marketing scenarios across platforms. Insights on writer scaling, concurrency control, and maintenance.
Honeywell Intelligrated's IoT Streaming Lakehouse for Warehouse Automation
Explore Honeywell Intelligrated's journey in launching IoT streaming services for warehouse automation, utilizing Databricks Lakehouse for data governance, processing, and AI-driven solutions.