Building a Lakehouse Architecture for Business Data Empowerment
Explore a 36-minute conference talk on building a data platform using the LakeHouse Architecture. Learn how Wehkamp created a uniform system to provide reliable, timely, and GDPR-compliant data access across their company. Discover the three-level data curation approach - bronze, silver, and gold - and how it enables data democratization while maintaining privacy. Gain insights into the use of open-source technologies, pseudonymization of PII fields, and the development of a custom library for efficient data source ingestion. Understand the implementation of key components such as ACID transactions, Structured Stream processing, Slack alerting, data quality checks, and CI/CD. Hear about the platform's positive impact on various teams and its role in modernizing Wehkamp's data infrastructure.