Главная
Study mode:
on
You
History
Saved
In progress 0 courses
compleat 0 courses
#Art & Design #Adobe #ChatGPT #GitHub

#Apache Spark

#Spark Streaming #Delta Lake #Spark SQL #Spark Structured Streaming #MLlib
Showing: 520 courses
NPTEL

Big Data Computing

13 rewiews
Comprehensive exploration of big data technologies, covering Hadoop, Spark, key-value stores, streaming analytics, machine learning, and graph processing for large-scale data analysis and insights.
Add to list
35 Lesons
19 hours
On-Demand
Free-Video
MLOps.community

Building Data Infrastructure at Scale for AI/ML with Open Data Lakehouses

0 rewiews
Explore how data lakehouse architecture with Apache Hudi supports real-world predictive ML and vector-based AI use cases in this 30-minute keynote by Vinoth Chandar, creator of Apache Hudi. Learn about ingesting data with minute-level freshness, providin…
Add to list
1 Lesons
30 minutes
On-Demand
Free-Video
CodeWithYu

Realtime Streaming for Anomaly Detection - An End-to-End Data Engineering Project

0 rewiews
Build a real-time anomaly detection system using streaming data engineering techniques. Learn end-to-end implementation for monitoring and identifying unusual patterns in data streams.
Add to list
1 Lesons
1 hour 6 minutes
On-Demand
Free-Video
MLOps World: Machine Learning in Production

Systematic Approaches and Creativity - Building DoorDash's ML Platform During the Pandemic

0 rewiews
Explore DoorDash's ML platform development journey, focusing on collaboration, guardrails, principled approaches, and architecture powering billions of daily predictions for diverse use cases.
Add to list
1 Lesons
36 minutes
On-Demand
Free-Video
GOTO Conferences

Building a CML Pipeline with Spark and Kafka - YOW! 2018

0 rewiews
Explore building a CML pipeline using Apache Spark and Kafka. Learn functional programming techniques for efficient data processing and machine learning workflows.
Add to list
1 Lesons
47 minutes
On-Demand
Free-Video
MLOps World: Machine Learning in Production

The Who, What, and Why of Data Lake Table Formats

0 rewiews
Explore data lake table formats, their creators, purposes, and significance in modern data architecture. Gain insights into optimizing data management and analytics.
Add to list
1 Lesons
48 minutes
On-Demand
Free-Video
Cloudera

How to Use Cloudera Data Platform

0 rewiews
Comprehensive guide to Cloudera Data Platform, covering key components, features, and practical applications for data management, analytics, and machine learning in cloud environments.
Add to list
42 Lesons
2 hours 30 minutes
On-Demand
Free-Video

Cloudera Certified Associate Spark and Hadoop Developer

0 rewiews
Comprehensive training for Cloudera Certified Associate Spark and Hadoop Developer certification, covering Sqoop, Flume, HDFS, Spark, and Hive. Hands-on practice with real-world scenarios and data analysis techniques.
Add to list
94 Lesons
1 day 8 hours
On-Demand
Free-Video
Great Learning

Spark Streaming Tutorial - Twitter Real Time Streaming - Apache Spark for Beginners

0 rewiews
Apache Spark Streaming is a scalable fault-tolerant streaming processing system that natively supports both batch and streaming workloads. This video will help you to understand every aspect of Spark Streaming along with the examples and visuals for bett…
Add to list
6 Lesons
2 hours 35 minutes
On-Demand
Free-Video
Great Learning

Apache Spark Full Course - Learn Apache Spark 2020

0 rewiews
In this 'Spark Tutorial' you will comprehensively learn all the major concepts of Spark such as Spark RDD, Dataframes, Spark SQL and Spark Streaming.
Add to list
14 Lesons
7 hours 44 minutes
On-Demand
Free-Video
Great Learning

Big Data Analytics for Beginners

0 rewiews
Comprehensive introduction to big data analytics, covering key concepts like Hadoop, Apache Spark, and ETL. Ideal for beginners seeking to understand the landscape and value chain of big data.
Add to list
8 Lesons
1 hour 11 minutes
On-Demand
Free-Video

Data Science Projects

0 rewiews
Hands-on projects covering deep learning, machine learning, and data analysis. Practical applications in healthcare, finance, and more using Python, TensorFlow, and other tools.
Add to list
26 Lesons
6 hours
On-Demand
Free-Video

PySpark with Python

0 rewiews
Learn PySpark for big data processing: DataFrame operations, handling missing values, filtering, aggregations, MLlib basics, and Databricks integration. Includes linear regression implementation.
Add to list
9 Lesons
2 hours 30 minutes
On-Demand
Free-Video

Spark Tutorials for Beginners

0 rewiews
Learn Apache Spark fundamentals, including RDDs, SQL, streaming, and architecture. Hands-on tutorials cover setup, programming, and advanced concepts for big data processing.
Add to list
17 Lesons
5 hours
On-Demand
Free-Video
Learning Journal

Scala Tutorial

0 rewiews
Comprehensive Scala programming tutorial for Spark developers and functional programmers, covering basics, concepts, and examples to prepare for Apache Spark development.
Add to list
35 Lesons
4 hours 30 minutes
On-Demand
Free-Video
Learning Journal

Apache Spark Tutorials

0 rewiews
Learn Apache Spark's distributed computing capabilities, architecture, and programming with Scala. Explore DataFrames, SQL, data sources, and advanced features like Delta Lake for efficient big data processing.
Add to list
21 Lesons
5 hours
On-Demand
Free-Video
Learning Journal

Apache Spark 3 - Beyond Basics and Cracking Job Interviews

0 rewiews
Advanced Apache Spark 3 skills for certification, job interviews, and complex scenarios. Covers architecture, memory management, optimization techniques, and performance tuning. Open to learner requests for additional topics.
Add to list
24 Lesons
3 hours 30 minutes
On-Demand
Free-Video

Apache Spark, Hadoop Project with Kafka and Python - End to End Development

0 rewiews
Comprehensive big data project integrating Apache Spark, Hadoop, Kafka, and Python for end-to-end development, including streaming, REST API, and dashboard creation.
Add to list
8 Lesons
1 hour 30 minutes
On-Demand
Free-Video
The AI University

Machine Learning Using Spark MLLib

0 rewiews
Learn to implement machine learning algorithms using Spark MLLib, covering linear regression techniques, data preprocessing, model building, and evaluation methods in a hands-on, practical approach.
Add to list
13 Lesons
2 hours 30 minutes
On-Demand
Free-Video
Databricks

Tech Talk - Top Tuning Tips for Spark 3.0 and Delta Lake on Databricks

0 rewiews
Learn top tuning tips for Apache Spark 3.0 and Delta Lake on Databricks. Discover optimal join strategies, machine sizing, merge operation speedups, and job simplification techniques for enhanced big data processing.
Add to list
10 Lesons
52 minutes
On-Demand
Free-Video
The ASF

Optimization for Reducing Compute Cost of SparkSQL in ByteDance

0 rewiews
Discover key optimization techniques for SparkSQL implementation at ByteDance, focusing on compute cost reduction strategies and performance enhancement methods.
Add to list
1 Lesons
33 minutes
On-Demand
Free-Video
Scala Days Conferences

Diamond Architecture on Scala 3 - Managing Large Codebases

0 rewiews
Explore Diamond Architecture in Scala 3 for managing large codebases. Learn OOP techniques, modular approaches, and new Scala 3 features to build maintainable applications.
Add to list
1 Lesons
41 minutes
On-Demand
Free-Video
CNCF [Cloud Native Computing Foundation]

Running Remote Shuffle Service to Solve Apache Spark's Dynamic Resource Allocation Challenge on Kubernetes

0 rewiews
探讨利用远程洗牌服务解决Apache Spark在Kubernetes上的动态资源分配挑战,实现存储与计算解耦,提高大数据处理的可靠性和可扩展性。
Add to list
1 Lesons
54 minutes
On-Demand
Free-Video
Ubuntu OnAir

Crea tu Propio Laboratorio de Big Data con MicroK8s y Spark

0 rewiews
Aprende a crear un laboratorio de Big Data utilizando Spark y MicroK8s. Descubre cómo emular una arquitectura de datos y practica habilidades esenciales para el análisis de grandes volúmenes de información.
Add to list
1 Lesons
57 minutes
On-Demand
Free-Video
The ASF

Bytedance's Parquet Format Optimization for Cost Reduction and Efficiency

0 rewiews
Optimize Parquet file operations for big data efficiency. Learn binary copy techniques to enhance file overwriting, merge small files, and implement column-level TTL, boosting performance 10+ times.
Add to list
1 Lesons
23 minutes
On-Demand
Free-Video
Databricks

Accelerating Real-Time Video Analytics on a Heterogeneous CPU and FPGA Platform

0 rewiews
Explore real-time video analytics acceleration using CPU+FPGA platforms. Learn about performance optimization, latency reduction, and cost-effective solutions for industrial applications like fraud detection and surveillance.
Add to list
20 Lesons
38 minutes
On-Demand
Free-Video
Databricks

Scaling Security Threat Detection with Apache Spark and Databricks

0 rewiews
Explore advanced security threat detection using Apache Spark and Databricks. Learn about innovative solutions for notebook-based testing, self-tuning alerts, automated investigations, and custom SDK development for enhanced cybersecurity.
Add to list
26 Lesons
24 minutes
On-Demand
Free-Video
Databricks

Data Privacy Techniques with Apache Spark - Defensive and Offensive Approaches

0 rewiews
Explore data privacy techniques for Apache Spark, including offensive and defensive approaches, k-anonymity, and quasi-identifiers. Learn implementation strategies and how Databricks Delta enhances dataset privacy.
Add to list
18 Lesons
27 minutes
On-Demand
Free-Video
Databricks

Scaling Privacy in a Spark Ecosystem

0 rewiews
Learn strategies for scaling privacy in Spark ecosystems, balancing customer rights with business needs. Explore centralized access control, auditing, and reporting in open data environments.
Add to list
9 Lesons
26 minutes
On-Demand
Free-Video
OWASP Foundation

Devil in the Haystack - Application Security Through Statistical Analysis and Machine Learning

0 rewiews
Exploring statistical and machine learning techniques for in-app security at Salesforce. Focuses on adaptive, data-driven approaches for detecting and preventing attacks and abuses on the platform.
Add to list
1 Lesons
52 minutes
On-Demand
Free-Video
Linux Foundation

Building a Flourishing Technical Community - Lessons from Delta Lake

0 rewiews
Discover key strategies for building and growing a thriving technical community, focusing on developer engagement, branding, collaboration, and effective management with Linux Foundation partnership.
Add to list
1 Lesons
30 minutes
On-Demand
Free-Video
Databricks

Becoming GDPR and CCPA Compliant Using Delta Live Tables

0 rewiews
Explore solutions for implementing "Right to be Forgotten" in data lakehouses using Delta Live Tables, ensuring compliance with GDPR and CCPA privacy regulations.
Add to list
1 Lesons
19 minutes
On-Demand
Free-Video
Databricks

Ensuring GDPR Compliance for Real-Time Data Pipelines

0 rewiews
Strategies for GDPR compliance in real-time data pipelines using Databricks tools. Covers immediate solutions with Delta Lake's CDF and long-term plans for PII separation, offering practical privacy compliance approaches.
Add to list
1 Lesons
31 minutes
On-Demand
Free-Video
OSACon

Unlocking Scalable and Efficient Data Storage with Apache Ozone

0 rewiews
Discover how Apache Ozone revolutionizes distributed object storage, offering scalable solutions for managing massive data volumes while ensuring reliability, cost-effectiveness, and seamless integration with Hadoop ecosystem.
Add to list
1 Lesons
27 minutes
On-Demand
Free-Video
Open Data Science

State of the Art Natural Language Understanding at Scale - David Talby

0 rewiews
Explore state-of-the-art NLP techniques using Apache Spark, covering core algorithms, distributed pipelines, and performance optimizations for large-scale language understanding applications.
Add to list
23 Lesons
50 minutes
On-Demand
Free-Video
Open Data Science

How Companies Are Using Tachyon, a Memory Centric Distributed Storage

0 rewiews
Explore Tachyon, a memory-centric distributed storage system for fast big data processing. Learn about its benefits, use cases, and integration with popular frameworks like Spark and Hadoop.
Add to list
13 Lesons
22 minutes
On-Demand
Free-Video
Databricks

Funnel Analysis with Apache Spark and Druid for Advertising Campaign Effectiveness

0 rewiews
Learn to perform funnel analysis at scale using Apache Spark, Druid, and DataSketches. Discover techniques for measuring campaign effectiveness and analyzing user behavior in chronological order for large-scale advertising campaigns.
Add to list
24 Lesons
26 minutes
On-Demand
Free-Video
Databricks

Degrading Performance - Understanding and Solving Small Files Syndrome

0 rewiews
Learn how small files impact data pipeline performance, understand cloud storage and Apache Spark interactions, and discover DeltaLake solutions for improved efficiency in handling massive datasets.
Add to list
9 Lesons
27 minutes
On-Demand
Free-Video
Databricks

Turbocharging Spark with Samsung SmartSSD Computational Storage Devices

0 rewiews
Discover how Samsung SmartSSD® with Xilinx FPGAs accelerates Spark, offering 2x-8x performance gains without code changes. Learn about the technology, benchmarks, and TCO savings.
Add to list
13 Lesons
30 minutes
On-Demand
Free-Video
Databricks

Photon: Accelerating SQL Queries on Databricks Lakehouse Platform

0 rewiews
Explore Photon, a C++ component for efficient query execution in Databricks Lakehouse. Learn about its vectorized processing, compatibility with Apache Spark, and performance enhancements for data-driven decision-making.
Add to list
12 Lesons
30 minutes
On-Demand
Free-Video
Databricks

Koalas: Scaling Pandas APIs on Apache Spark - Performance and Comparison with Dask

0 rewiews
Explore Koalas: pandas-like APIs for big data on Apache Spark. Compare performance with Dask, learn about recent updates, and understand its scalability for data science tasks.
Add to list
11 Lesons
24 minutes
On-Demand
Free-Video
Databricks

Empowering Developers with Self-Service ETL - Zillow's Approach

0 rewiews
Zillow's self-service ETL solutions empower teams to build, maintain, and monitor data pipelines, abstracting complex processes and catering to diverse user needs while leveraging internal services for efficient implementation.
Add to list
17 Lesons
24 minutes
On-Demand
Free-Video
Databricks

Advancing GPU Analytics with RAPIDS Accelerator for Apache Spark and Alluxio

0 rewiews
Learn to accelerate GPU analytics using RAPIDS for Apache Spark and Alluxio, enabling faster data access and processing for AI and analytics workloads across any cloud environment.
Add to list
15 Lesons
27 minutes
On-Demand
Free-Video
Databricks

A Collaborative Data Science Development Workflow Using Kedro and MLflow

0 rewiews
Explore a collaborative data science workflow using Kedro pipelines, MLflow tracking, and cloud-agnostic GPU containers for efficient model development, training, and production deployment.
Add to list
13 Lesons
24 minutes
On-Demand
Free-Video
Databricks

Growing the Delta Ecosystem - Delta-RS for Rust and Python

0 rewiews
Explore delta-rs, a Rust library extending Delta Lake beyond Spark. Learn its current capabilities, roadmap, and real-world application in kafka-delta-ingest for high-throughput data transfer.
Add to list
11 Lesons
25 minutes
On-Demand
Free-Video
Databricks

Building Lakehouses on Delta Lake with SQL Analytics

0 rewiews
Learn to build lakehouses using Delta Lake, perform SQL analytics, and connect BI tools. Explore data management techniques, governance models, and implementation examples for efficient data processing and analysis.
Add to list
28 Lesons
31 minutes
On-Demand
Free-Video
Databricks

Make Reliable ETL Easy on Delta Lake

0 rewiews
Learn to simplify ETL development on Delta Lake, improving data quality and scaling operations. Discover how Databricks streamlines the ETL lifecycle for efficient data engineering and management.
Add to list
14 Lesons
45 minutes
On-Demand
Free-Video
Databricks

Delta Lake Streaming - Internals and Query Progress Logs

0 rewiews
Explore Delta Lake streaming internals, focusing on structured streaming components, Query Progress Logs, and checkpoint directories for efficient real-time data pipeline management.
Add to list
9 Lesons
29 minutes
On-Demand
Free-Video
Databricks

Building Data Quality Pipelines with Apache Spark and Delta Lake

0 rewiews
Learn to build robust data quality pipelines using Apache Spark and Delta Lake. Explore rule templates, reporting models, and PowerBI integration for effective data remediation and self-healing in enterprise environments.
Add to list
8 Lesons
27 minutes
On-Demand
Free-Video
Databricks

Why Lakehouse Architecture Now - Exploring Enterprise Data Warehouse Failures and the Need for Lakehouse Paradigm

0 rewiews
Explore the shift from enterprise data warehouses to lakehouse architecture, focusing on benefits, challenges, and Starburst's role in powering SQL-based analytics on Delta Lake.
Add to list
16 Lesons
26 minutes
On-Demand
Free-Video
Databricks

Creating a Lakehouse on AWS

0 rewiews
Learn to build a Lakehouse on AWS, combining data lake affordability with data warehouse performance. Discover integration with AWS services and real-world examples of high-performance, cost-effective analytics solutions.
Add to list
8 Lesons
22 minutes
On-Demand
Free-Video
Databricks

Building End-to-End Delta Pipelines on GCP

0 rewiews
Explore building end-to-end Delta pipelines on GCP, covering data reliability, performance, and the Bronze-Silver-Gold architecture pattern, with practical code examples and Big Query integration.
Add to list
11 Lesons
27 minutes
On-Demand
Free-Video
Databricks

Apache Spark, Data Science, and Machine Learning Advances - Data + AI Summit 2021 Keynote

0 rewiews
Explore AI advancements, data science, MLOps, and Databricks Lakehouse Platform innovations. Featuring industry leaders and Bill Nye, this keynote covers MLFlow updates, AutoML, and practical machine learning solutions.
Add to list
24 Lesons
2 hours 18 minutes
On-Demand
Free-Video
Databricks

Boost Delta Lake Performance with Data Skipping and Z-Order

0 rewiews
Explore data skipping and Z-ordering techniques to enhance Delta Lake performance, focusing on optimized partitioning strategies for improved read and write efficiency in large-scale data lakes.
Add to list
17 Lesons
41 minutes
On-Demand
Free-Video
Databricks

How Apache Spark 3.0 and Delta Lake Enhance Data Lake Reliability

0 rewiews
Discover performance enhancements in Apache Spark 3.0 and Delta Lake's data reliability features. Learn about Adaptive Query Execution, Dynamic Partition Pruning, and handling skewed queries for improved big data processing.
Add to list
39 Lesons
57 minutes
On-Demand
Free-Video
Databricks

Engagement Activity Delta Lake for Einstein Analytics and Sales Cloud Einstein

0 rewiews
Explore how Salesforce built an engagement activity platform using Delta Lake to support Einstein Analytics and Sales Cloud Einstein, covering data ingestion, incremental reads, and mutation handling.
Add to list
26 Lesons
51 minutes
On-Demand
Free-Video
Databricks

MLflow Model Serving - From Training to Production

0 rewiews
Comprehensive overview of MLflow Model Serving, covering offline and online scoring, deployment options, and integration with Databricks, with practical examples and recent features.
Add to list
31 Lesons
37 minutes
On-Demand
Free-Video
Databricks

Developing Scalable Machine Learning Pipelines for Gaming Industry

0 rewiews
Explore scalable ML pipelines in gaming, leveraging Apache Spark for data-driven product development and improved user acquisition through advanced LTV modeling.
Add to list
23 Lesons
29 minutes
On-Demand
Free-Video
Databricks

Accelerating MLOps with Apache Spark and SingleStore

0 rewiews
Learn to accelerate MLOps using Apache Spark and SingleStore. Discover how to build fast analytical applications, manage supervised learning models, and scale AI/ML operations for data science success.
Add to list
10 Lesons
25 minutes
On-Demand
Free-Video
Databricks

Cost Efficiency Strategies for Managed Apache Spark Service with Databricks

0 rewiews
Explore cost-efficient strategies for Azure Databricks, including resource management, tier selection, and best practices for optimizing Apache Spark workloads in a managed cloud environment.
Add to list
7 Lesons
26 minutes
On-Demand
Free-Video
Databricks

Migrating Airflow-Based Apache Spark Jobs to Kubernetes - The Native Way

0 rewiews
Learn to migrate Airflow-based Apache Spark jobs to Kubernetes, reducing costs and improving stability. Explore challenges, best practices, and native Airflow integration for seamless transition from AWS EMR to K8s.
Add to list
19 Lesons
21 minutes
On-Demand
Free-Video
Databricks

Scaling Machine Learning Feature Engineering in Apache Spark at Facebook

0 rewiews
Explore advanced techniques for scaling ML feature engineering in Apache Spark, including Feature Injection and Feature Reaping, with optimizations for improved performance and efficiency at Facebook's scale.
Add to list
13 Lesons
21 minutes
On-Demand
Free-Video
Databricks

Project Zen - Improving Apache Spark for Python Users

0 rewiews
Discover improvements in Apache Spark for Python users, including redesigned documentation, type hints, PyPI distribution options, and enhanced PySpark features for a more Pythonic experience.
Add to list
19 Lesons
21 minutes
On-Demand
Free-Video
Databricks

Getting Started with Apache Spark on Kubernetes

0 rewiews
Learn to deploy and optimize Apache Spark on Kubernetes with hands-on demos, covering environment setup, application sizing, performance tuning, and monitoring for efficient data pipeline management.
Add to list
10 Lesons
26 minutes
On-Demand
Free-Video
Databricks

Delta Lake: Optimizing Merge Operations

0 rewiews
Explore Delta Lake merge optimization techniques, including inner and full outer joins, partition pruning, and file pruning. Learn tips for handling large merges efficiently.
Add to list
9 Lesons
24 minutes
On-Demand
Free-Video
Databricks

Building a Cloud Data Lake with Databricks and AWS - Best Practices and Implementation

0 rewiews
Explore building a cloud data lake using Databricks and AWS. Learn about data lake architecture, AWS services, Delta Lake for reliability and performance, and best practices for implementation.
Add to list
19 Lesons
29 minutes
On-Demand
Free-Video
Databricks

Stateful Streaming with Apache Spark - Updating Decision Logic at Runtime

0 rewiews
Learn how to implement stateful streaming with Spark, focusing on updating decision logic at runtime for real-time offer personalization in loyalty programs like PAYBACK.
Add to list
6 Lesons
25 minutes
On-Demand
Free-Video
Databricks

Leveraging Apache Spark and Delta Lake for Efficient Data Encryption at Scale

0 rewiews
Efficient data encryption solution for privacy compliance using Apache Spark and Delta Lake. Automates consumer deletion requests, enhances security, and maintains data integrity while ensuring accessibility when needed.
Add to list
13 Lesons
25 minutes
On-Demand
Free-Video
Databricks

Common Strategies for Improving Performance on Delta Lakehouse

0 rewiews
Explore techniques to enhance Delta Lakehouse query performance, including data skipping, optimizing Z-Order, SQL partitions, and Delta Cache. Learn to analyze and tune queries for better results.
Add to list
10 Lesons
31 minutes
On-Demand
Free-Video
Databricks

Diving into Delta Lake: Understanding the Transaction Log

0 rewiews
Explore Delta Lake's transaction log: its structure, functionality, and role in ACID transactions, metadata handling, time travel, and concurrent operations. Gain insights into this core component.
Add to list
16 Lesons
30 minutes
On-Demand
Free-Video
Databricks

Optimizing Geospatial Queries with Dynamic File Pruning in Databricks Delta

0 rewiews
Optimize geospatial queries using z-ordering and dynamic file pruning techniques in Databricks Delta, significantly improving query times for petabyte-scale data through specific data generation and SQL query design strategies.
Add to list
23 Lesons
25 minutes
On-Demand
Free-Video
Databricks

Demystifying Delta Lake - Data Reliability for Data Lakes

0 rewiews
Explore Delta Lake's reliability features for data lakes with Apache Spark creator Michael Armbrust. Learn about ACID transactions, metadata handling, and unified data processing.
Add to list
18 Lesons
26 minutes
On-Demand
Free-Video
Databricks

SQL Analytics and the Lakehouse Architecture - Data + AI Summit EU 2020 Keynote

0 rewiews
Explore lakehouse architecture, Delta Lake, and SQL Analytics for unified data management. Learn how this approach combines data warehouse capabilities with data lake flexibility, enabling faster analytics and decision-making.
Add to list
16 Lesons
54 minutes
On-Demand
Free-Video
Databricks

How Apache Spark 3.0 and Delta Lake Enhance Data Lake Reliability

0 rewiews
Explore Apache Spark 3.0 and Delta Lake's enhancements for data lake reliability, including ACID transactions, schema enforcement, and time travel. Learn about performance improvements and new features in Databricks Runtime 7.0 Beta.
Add to list
23 Lesons
58 minutes
On-Demand
Free-Video
Databricks

Fugue: Unifying Big Data Analytics Ecosystems for ETL and Machine Learning

0 rewiews
Unifying big data ecosystems with Fugue: SQL-like framework for ETL and ML pipelines, compatible with Spark, TensorFlow, and more. Simplifies development, improves performance, and enhances maintainability.
Add to list
13 Lesons
22 minutes
On-Demand
Free-Video
Databricks

Presto on Apache Spark - A Tale of Two Computation Engines

0 rewiews
Explore the integration of Presto and Spark, combining low-latency evaluation with robust execution for enhanced SQL experience in both interactive and batch use cases at Facebook scale.
Add to list
17 Lesons
25 minutes
On-Demand
Free-Video
Databricks

Designing the Next Generation of Data Pipelines with Apache Spark - Zillow's Approach

0 rewiews
Zillow's data engineering team shares their journey in redesigning data pipelines using Apache Spark, focusing on scalability, maintainability, and robustness while reducing code complexity.
Add to list
9 Lesons
27 minutes
On-Demand
Free-Video
Databricks

Simplify and Boost Spark 3 Deployments with Hypervisor-Native Kubernetes

0 rewiews
Explore integrating Spark 3 with VMware Pacific for scalable, reliable, and high-performance big data analytics deployments using hypervisor-native Kubernetes on vSphere 7.
Add to list
7 Lesons
33 minutes
On-Demand
Free-Video
Databricks

Optimizing Catalyst Optimizer for Complex Spark Plans

0 rewiews
Optimizing Spark's Catalyst optimizer for complex plans with thousands of operators and large case statements. Techniques include memory-efficient logging, subexpression elimination, and improved constraint propagation.
Add to list
22 Lesons
27 minutes
On-Demand
Free-Video
Databricks

Natural Language Query and Conversational Interface to Apache Spark

0 rewiews
Explore conversational analytics with Apache Spark, enabling natural language queries for data insights. Learn about NLP-to-SQL translation and its integration with dialog management and visualization.
Add to list
12 Lesons
25 minutes
On-Demand
Free-Video
Databricks

Modernizing Apache Spark 3.0 Applications - Best Practices and Optimization Techniques

0 rewiews
Optimize legacy Spark applications for 3.0: leverage Adaptive Query Execution, columnar processing, GPU acceleration, and ML integration. Concrete examples and lessons from modernizing analytics workloads.
Add to list
15 Lesons
25 minutes
On-Demand
Free-Video
Databricks

Learn to Use Databricks for Data Science

0 rewiews
Discover how Databricks simplifies data science with a unified platform for analytics workloads, from data preparation to predictive analytics, enhancing collaboration and productivity at scale.
Add to list
23 Lesons
39 minutes
On-Demand
Free-Video
Databricks

Hybrid Apache Spark Architecture: Optimizing YARN and Kubernetes for Lyft's Workloads

0 rewiews
Explore Lyft's hybrid Apache Spark architecture using YARN and Kubernetes, addressing scalability challenges and optimizing workload management for improved reliability and efficiency.
Add to list
16 Lesons
44 minutes
On-Demand
Free-Video
Databricks

Optimizing Spark SQL Jobs with Parallel and Asynchronous IO

0 rewiews
Optimizing Spark SQL jobs through parallel and async IO techniques, including file-level and row group-level parallel read, asynchronous spill, and parquet column family, resulting in 5-30% performance improvement.
Add to list
8 Lesons
21 minutes
On-Demand
Free-Video
Databricks

Fully Utilizing Spark for Data Validation with Fugue and Pandera

0 rewiews
Explore lightweight data validation for Spark using Fugue and Pandera, enabling partition-specific rules and efficient big data processing without compromising on functionality.
Add to list
12 Lesons
22 minutes
On-Demand
Free-Video
Databricks

Drug and Vaccine Discovery Using Knowledge Graphs and Apache Spark

0 rewiews
Explore how GSK uses knowledge graphs and Apache Spark to build a massive medical database, enabling advanced drug and vaccine discovery through innovative data querying and machine learning techniques.
Add to list
14 Lesons
31 minutes
On-Demand
Free-Video
Databricks

Delight - Improved Apache Spark UI for Performance Troubleshooting

0 rewiews
Explore Delight, a free monitoring dashboard for Apache Spark. Learn to troubleshoot and optimize data engineering pipelines using system metrics and Spark information for improved performance and cost-effectiveness.
Add to list
5 Lesons
21 minutes
On-Demand
Free-Video
Databricks

Deep Dive into New Features of Apache Spark 3.1

0 rewiews
Explore Apache Spark 3.1's new features: ANSI SQL compliance, streaming enhancements, Python improvements, and performance optimizations for faster, easier, and smarter data processing.
Add to list
30 Lesons
49 minutes
On-Demand
Free-Video
Databricks

Data Quality Tools Comparison for Continuous Data Imports

0 rewiews
Comparison of open-source data quality tools for continuous imports, covering maturity, documentation, extensibility, and features like data profiling and anomaly detection.
Add to list
17 Lesons
28 minutes
On-Demand
Free-Video
Databricks

Configuration-Driven Reporting on Large Datasets Using Apache Spark

0 rewiews
Configurable enterprise framework for generating and distributing diverse reports from large financial datasets using Apache Spark, with dynamic scheduling and data transformation capabilities.
Add to list
12 Lesons
27 minutes
On-Demand
Free-Video
Databricks

Comprehensive View on Intervals in Apache Spark 3.2

0 rewiews
Explore Apache Spark 3.2's interval enhancements, including ANSI SQL conformance, new interval types, and improved APIs. Learn to construct, use, and manage intervals effectively in Spark SQL and PySpark.
Add to list
14 Lesons
25 minutes
On-Demand
Free-Video
Databricks

Build Real-Time Applications with Databricks Streaming - Fire Department Use Case

0 rewiews
Learn to build real-time applications using Databricks Streaming, focusing on a fire department use case. Explore architecture, data sources, and implementation for live tracking of equipment, personnel, and incidents.
Add to list
12 Lesons
30 minutes
On-Demand
Free-Video
Databricks

Migrating ETL Workflows to Apache Spark at Scale - Pinterest's Experience

0 rewiews
Explore Pinterest's journey migrating ETL workflows to Apache Spark, covering challenges, solutions, and performance improvements. Learn about semantic gaps, thrift objects, and innovative Spark profiling techniques.
Add to list
23 Lesons
25 minutes
On-Demand
Free-Video
Databricks

Managing Millions of Tests Using Databricks - Automated Monitoring and Reporting System

0 rewiews
Explore automated test monitoring and reporting for Databricks Runtime using Delta, analyzing results from various sources to track quality and efficiently report failures to owners.
Add to list
14 Lesons
25 minutes
On-Demand
Free-Video
Databricks

Large Scale Geospatial Indexing and Analysis on Apache Spark

0 rewiews
Explore large-scale geospatial indexing and analysis using Apache Spark, covering challenges, frameworks, data structures, and practical applications in cloud-first architecture with Databricks.
Add to list
11 Lesons
23 minutes
On-Demand
Free-Video
Databricks

Delta Lake 2.0 Overview - New Features and Community Collaborations

0 rewiews
Overview of Delta Lake 2.0 features, integrations, and community collaboration. Highlights include data skipping, Change Data Feed, column mapping, multi-cluster S3 writes, and support for various languages and platforms.
Add to list
15 Lesons
38 minutes
On-Demand
Free-Video
Databricks

Gazelle-JNI: Offloading Spark SQL to Native Engines for Execution Acceleration

0 rewiews
Explore Gazelle-Jni, a middle layer integrating native SQL engines with Spark SQL for accelerated execution through Substrait-transformed physical plan offloading.
Add to list
21 Lesons
41 minutes
On-Demand
Free-Video
Databricks

Enable Production ML with Databricks Feature Store

0 rewiews
Simplify production ML with Databricks Feature Store. Learn to overcome data challenges, enable real-time inference, and streamline ML projects using a unified data lakehouse platform integrated with Apache Spark and MLflow.
Add to list
14 Lesons
33 minutes
On-Demand
Free-Video
Databricks

Delta Live Tables: Modern Software Engineering for ETL Pipelines

0 rewiews
Learn how Delta Live Tables simplifies ETL, enhances data pipeline reliability, and automates tasks. Discover modern software engineering practices for efficient data transformation and improved analytics, data science, and ML outcomes.
Add to list
18 Lesons
36 minutes
On-Demand
Free-Video
Databricks

Recent Parquet Improvements in Apache Spark - Vectorized Complex Types and Column Index Support

0 rewiews
Explore recent improvements in Apache Spark's Parquet performance, including vectorized read support for complex types and column index support, leading to significant speed enhancements and efficient data processing.
Add to list
19 Lesons
37 minutes
On-Demand
Free-Video