Introduction to Databricks and Data Lakehouse Architecture
Explore Data Lakehouse, Databricks platform, Spark architecture, and SQL. Learn advantages and common use cases of Databricks for data processing.
Introduction to Data Build Tool (DBT)
Master data build tool (DBT) for efficient data product development, including CICD, pipelines, validation, quality testing, deployment, and monitoring.
Implementing Change Data Capture (CDC) and Slowly Changing Dimensions (SCD) Using Spark and Hive
Learn to implement Change Data Capture (CDC) and Slowly Changing Dimension (SCD) patterns in Hive using Spark, with a step-by-step guide for efficient data management and analysis.
How to Build Data Pipelines for ML Projects with Python Code
Learn to build data pipelines for machine learning projects using Python. Explore ETL vs ELT, extraction, transformation, loading, and orchestration. Includes practical example of processing YouTube video transcripts.
4 Skills Needed to Be a Full-Stack Data Scientist
Discover the four essential skills for full-stack data science, exploring project management, data engineering, data science, and machine learning engineering roles in this comprehensive overview.
Automating Data Pipelines with Python and GitHub Actions - Code Walkthrough
Automate data pipelines using Python and GitHub Actions. Learn to create ETL scripts, set up repositories, configure workflows, and implement ML applications for efficient data processing.
Practice of Building Real-Time Data Lake Based on Flink
Build a real-time data lake using Flink and Iceberg to address timeliness, diversity, consistency, and accuracy challenges in modern data architectures.
Challenges and Solutions for Building Real-time Data Warehousing with Apache Flink, Apache Hive and Apache Iceberg
Explore solutions for real-time data warehousing using Apache Flink, Hive, and Iceberg. Learn to handle challenges like late events and dirty data routing in enterprise-level ETL migration.
Introducing Apache Pinot: Real-time Analytics for Large-Scale Data
Explore Apache Pinot's real-time analytics capabilities, architecture, and data model. Learn about its query language, indexing, and use cases in high-performance applications.
Accelerating Big Data - Modern Trends, Enable Product Analytics
Explore modern trends in big data acceleration and enable effective product analytics for improved decision-making and business insights.
Apache Celeborn: Making Spark and Flink Faster, More Stable and More Resilient
Explore Apache Celeborn's high-performance Shuffle service for Spark and Flink, enhancing stability, performance, and cost-efficiency in large-scale data processing environments.
Building a Real-Time Analytics Database
Odkryj tajniki tworzenia bazy danych do analityki w czasie rzeczywistym. Poznaj kluczowe koncepcje i najlepsze praktyki w tej dziedzinie.
Cloud Agnostic Data Platform with DBT, Trino, Iceberg and MinIO
Explore a hands-on, cloud-agnostic data pipeline using TrinoDB, DBT, MinIO, and Apache Iceberg for scalable and lightweight analytics, suitable for both cloud and on-premise solutions.
Don't Fix Bad Data, Do This Instead - Pragmatic Strategies for Data Quality
Explore pragmatic strategies for bridging the gap between data expectations and reality. Learn about integration tests, prioritization, and balancing technical and cultural measures in data quality management.
DBT and Python - How to Write Reusable and Testable Pipelines
Develop reusable and testable data pipelines using DBT and Python. Learn best practices for version control, DAGs, and unit testing in SQL and Python-based models.
From Pandas to Production: ELT with dlt
Explore data load tool (dlt) to streamline data science workflows, overcome roadblocks, and transition smoothly from exploration to production. Learn best practices for data handling and managing failures with real-life examples.
Why ML in Production is Still Broken
Explore challenges in deploying ML projects to production, focusing on the disconnect between development and real-world application. Learn key aspects distinguishing ML from traditional software engineering.
Reaching Lightspeed Data Science - ETL, ML, and Graph with NVIDIA RAPIDS
Accelerate data science workflows using GPUs and RAPIDS. Learn performance gains, easy migration, and new possibilities in ETL, ML, and graph analytics.
Simplifying Machine Learning Lifecycle Management in Healthcare
Explore end-to-end ML lifecycle in healthcare using Azure services. Learn data ingestion, model creation, deployment, and visualization for real-time patient predictions and batch scoring.
MLOps Without Much Ops - Building Efficient Machine Learning Systems
Discover modern, no-nonsense data pipelines for efficient machine learning systems. Learn PaaS advantages and explore real-world applications with open-source code. Gain insights on ML's future for organizations of all sizes.
Retail Transactions at Scale for AI - From PoC to Production
Discover how to scale AI solutions for retail transactions, from handling millions of unstructured data points to enabling enterprise-wide AI applications and reporting capabilities.