Building a Data Platform for Mission-Critical Analytics on AWS with Databricks
Explore building enterprise data lakes on AWS with Databricks, complementing AWS strategy. Learn from HP's success in transforming business through unified data analytics and overcoming data lake challenges.
Slowly Changing Dimensions Type 2 in Data Lakes - Tech Chat
Explore Slowly Changing Dimensions Type 2 in data lakes using Delta Lake, focusing on market segmentation for a climbing eCommerce site. Learn to track historical and current data effectively.
How to Build a Cloud Data Platform - Part 1: Architecture
Learn to build a unified cloud data platform using Delta Lake, combining data warehouses and lakes for improved reliability, performance, and operations.
Building Data Intensive Analytic Applications on Top of Delta Lakes
Explore Delta Lake's ACID transactions for Apache Spark, enhancing data reliability and query performance. Learn to address challenges in modern data engineering through hands-on examples and practical applications.
Modern ETL Pipelines with Change Data Capture - Building Resilient Data Streams
Discover how GetYourGuide built a resilient ETL pipeline using Debezium, Kafka, Spark, and Airflow. Learn to handle schema changes automatically, stream data from SQL databases, and refresh data lakes multiple times daily for fresher insights.
Different Streaming Methods with Apache Spark and Kafka
Explore real-time analytics using Apache Spark and Kafka. Learn about streaming methods, overcoming technical challenges, and innovative solutions for handling billions of daily events in a scalable, cost-efficient manner.
Building the Petcare Data Platform with Delta Lake and Spark ETL Pipeline
Explore Mars Petcare's data platform using Delta Lake and Spark ETL pipeline 'Kyte'. Learn about advantages over Azure Data Factory and leveraging Delta Lake for ETL configurations and data science.
From BI to AI - Lakehouse Is the Modern Data Architecture - Vini Jaiswal
Explore Lakehouse, a modern data architecture combining data warehouses and lakes. Learn its principles, architecture, and real-world applications for efficient data management and analysis.
Hadoop Safari: Hunting for Vulnerabilities
Explore Hadoop security vulnerabilities, attack vectors, and tools for accessing data in Big Data clusters. Learn about design flaws and practical techniques to exploit weaknesses in Hadoop infrastructure.
Hail Hydrate - From Stream to Lake Using Open Source
Explore open-source tools for efficient data streaming and lake management, enhancing your skills in modern data architecture and processing techniques.
The Pace of Innovation in Delta Lake
Explore the rapid advancements and innovative features of Delta Lake, enhancing data reliability and performance in modern data architectures.
Delta Lake - Diving into Data Lakes Without the Downsides
Explore Delta Lake's innovative approach to data management, offering enhanced reliability and performance for large-scale analytics without traditional data lake drawbacks.
Big Data Outlook for 2017 and Why You Should Care
Explore key big data themes for 2017, strategies for addressing data needs, and success factors for implementing big data initiatives in various industries.
Generate Test Data Quick, Easy and in Large Volumes with the Databricks Labs Data Generator
Generate terabytes of proper test data quickly and easily using Databricks Labs Data Generator for dev environments across Azure, AWS, and GCP.
Caching Framework for Exabyte-Scale Data Lakes
Optimize exabyte-scale data lakes with open-source caching framework. Improve performance, reduce costs, and tackle challenges in complex data environments using Hadoop, Parquet, Hudi, and Alluxio.
Git for Data Lakes: How LakeFS Scales Data Versioning to Billions of Objects
Learn how lakeFS enables Git-like versioning for data lakes, solving challenges of object storage and scaling to billions of objects without compromising performance. Explore branching, committing, and merging for petabyte-scale data management.
OpenEuler and Bigtop with Ambari - Empowering Data Lake Solutions
Explore free open-source data lake solutions using Bigtop with Ambari. Learn about the latest Bigtop release, integration with OpenEuler, and how this platform streamlines data management and analytics.
Real-Time Data Integration Practice Based on Flink CDC at Alibaba Cloud
Explore real-time data integration using Flink CDC, covering core design, key implementations, and Alibaba Cloud's solutions for data lake and warehouse integration scenarios.
Gravitino - A Multi-Regional Geo-Distributed Meta Datalake
Discover how to build and manage distributed meta datalakes using Gravitino, exploring multi-regional architectures and implementation strategies for scalable data solutions.
Incremental Change Data Capture: A Data-Informed Journey
Explore data-driven strategies for incremental ingestion, from SaaS to databases and event streams. Learn cost-saving techniques, CDC flavors, and performance improvements for efficient data lake management.
Creating a Data and GenAI Ecosystem - Migrating to Databricks
Gerdau's journey to create a flexible, simplified Data and GenAI ecosystem using Databricks, resulting in cost reduction, tool consolidation, and improved governance for the steel industry.