Building Advance Analytics Pipelines with Azure Databricks
Explore Azure Databricks for advanced analytics: Learn Spark basics, collaboration features, cluster management, and Azure integration through a detailed network intrusion anomaly detection pipeline walkthrough.
Scale Data Science Using Pandas API on Spark
Discover how to scale data science workflows using Pandas API on Spark, enabling efficient processing of large datasets while maintaining familiar pandas functionality and optimized performance.
Navigating the Emerging LLMOps Stack
Explore the emerging LLMOps Stack, including tools and best practices for maximizing LLM potential in organizations. Gain insights from an experienced AI/ML engineering leader.
Developing Fast Applications With Open Source Software - Without the Fury
Explore performance and scalability challenges in building fast, scalable Big Data applications using open-source technologies like Apache Cassandra and Apache Kafka. Gain insights from real-world experiences.
Spark를 활용한 반도체 비정형 데이터 처리 및 조회 서비스 혁신
Discover how to optimize semiconductor unstructured data processing using Spark architecture and Big Data technologies for enhanced analysis capabilities in high-tech manufacturing environments.
Databricks Distributed File System (DBFS) vs Local Files on Spark Cluster Driver Node
Understand the key differences between Databricks File System (DBFS) and local driver node storage, exploring distributed storage capabilities and FUSE mount implementation for cloud data access.
Lakehouse Architecture with Delta Lake and Databricks
Dive into Lakehouse architecture fundamentals, exploring Delta Lake format and Databricks platform implementation for modern data management and analytics solutions.
REDLake Azure - Enterprise Data Lake and Business Satellites
Explore Bosch's REDLake Azure enterprise data repository system, focusing on data collection, storage, refinement, and analysis using Azure Synapse Analytics and Power BI for global accessibility.
Fixing Small Files Performance Issues in Apache Spark Using DataFlint
Master Apache Spark's file handling optimization techniques and learn to solve small files performance issues using DataFlint. Explore best practices for storage formats like Delta Lake and Iceberg.
Automated Translation of Functional Big Data Queries to SQL
Explore automated translation of functional big data queries to SQL, improving efficiency while maintaining abstraction benefits. Learn about novel column-wise decomposition for query synthesis.
Chidori - AI/ML Cluster Management Platform
Explore Chidori, an AI/ML cluster management platform for Spark on Kubernetes. Learn advantages, challenges, and see a demo of this innovative solution for efficient resource utilization.
Enriching the Data vs Filtering in Apache Spark
Explore data enrichment vs filtering in Spark for improved performance and scalability in cloud-native applications, with practical examples and insights from Capital One's loyalty use case.
Whitefox: Simple Table Format Sharing for Open Data Systems
Simplify data sharing with Whitefox, an open-source protocol for streamlined access management in table-format architectures, offering lightweight and native accessibility for Iceberg and Delta Lake.
Modern Data Orchestration: Best Practices and Real-World Use Cases
Elevate your data pipeline game with real-world tricks and best practices. Explore patterns, avoid dependencies, and integrate fundamental concepts for efficient data orchestration.
Govern Your Self-Service Integration Processes
Implement clear governance for self-service data integration across diverse environments. Control and audit data loading, processing, and uploading with WOODY.IO.
Creating and Submitting Scala Jobs to Spark Clusters Using Airflow
Build end-to-end data engineering project using Airflow, Docker, Spark, Scala, Python, and Java. Create and submit jobs to Spark clusters, process data, and view real-time results.
Creating and Submitting PySpark Jobs to Spark Clusters
Create and submit PySpark jobs to Spark clusters in an end-to-end data engineering project using Airflow, Docker, and multiple programming languages. Learn job creation, cluster submission, and result visualization.
End-to-End Data Engineering with Apache Airflow, Docker, and Spark Clusters - Using Python, Scala, and Java
Build an end-to-end data engineering project using Apache Airflow, Docker, Spark Clusters, Scala, Python, and Java. Create and submit jobs in multiple languages to a Spark cluster for processing and real-time results.
Creating and Submitting Java Spark Jobs to Spark Clusters
Create and submit Java Spark jobs to clusters. Learn end-to-end data engineering with Airflow, Docker, Spark, Scala, Python, and Java. Build and process jobs in multiple languages for live results.
Beyond Environment Variables: A Deep Dive into Efficient Configuration Management
Explore efficient configuration management strategies for secure, scalable infrastructure. Learn to streamline config updates, version control, and secret management across environments.
Scaling Data Analytics Securely with Spark on Google Cloud
Simplify and scale data analytics securely with Spark on Google Cloud, combining open source and cloud-native tools in a unified, fully managed platform for data and AI.