Using SQL Views in Databricks and Apache Spark - Lesson 12
Learn to create and utilize SQL views in Databricks for efficient data analysis and pipeline reuse. Explore common table expressions, aggregations, and case statements for advanced querying techniques.
Using the Data Science Process - Lesson 5
Explore the data science process from project inception to completion, covering key steps for successful implementation in Databricks and Apache Spark projects.
Create a Spark Cluster - Master Databricks and Apache Spark Step by Step - Lesson 4
Learn to create and use Apache Spark clusters, understand their differences from Databricks, and explore data querying with Zeppelin notebooks in this step-by-step guide.
Databricks Workspace Tour and Notebook Demo - Lesson 3
Comprehensive tour of Databricks Workspace, demonstrating tools and features with focus on the proprietary notebook IDE. Provides foundation for advanced Databricks and Apache Spark usage.
Create a Databricks Workspace - Lesson 2
Learn to create a Databricks workspace in Azure, understanding its components, pricing, and setup process. Gain practical skills for setting up your cloud-based data analytics environment.
Hitting and Resolving Azure Resource Limits
Learn to manage Azure resource limits, prevent deployment failures, and ensure smooth enterprise operations by understanding quota limits and how to increase them.
Databricks - How to Use Workflows
Learn to orchestrate and schedule complex multi-task processes using Databricks workflows, a powerful extension to the Jobs service that utilizes DAG architecture for efficient data processing and automation.
Pandas UDFs and Python Type Hints in Apache Spark 3.0
Explore redesigned pandas UDFs in Apache Spark 3.0, leveraging Python type hints for clearer, more Pythonic data science workflows and improved static analysis capabilities.
Apache Spark vs Databricks - On-Premise Performance Comparison
Comparison of on-premise Databricks and open-source Spark implementations, showcasing 10x performance gains in cyber workloads and addressing challenges of air-gapped analytics solutions.
Hyperspace - An Indexing Subsystem for Apache Spark
Explore Hyperspace, an indexing subsystem for Apache Spark that enhances query performance and reduces resource costs. Learn about its design, implementation, and benefits for data analytics on large datasets.
AutoML Toolkit Deep Dive - Automating Feature Engineering and Model Optimization
Explore Databricks Labs' AutoML Toolkit for automating feature engineering, model selection, tuning, and deployment. Learn to accelerate machine learning workflows and enhance productivity.
Building a Data Platform for Mission-Critical Analytics on AWS with Databricks
Explore building enterprise data lakes on AWS with Databricks, complementing AWS strategy. Learn from HP's success in transforming business through unified data analytics and overcoming data lake challenges.
Preventing Common Pitfalls in Production Streaming Jobs
Explore key strategies for optimizing streaming jobs in production, covering input parameters, stateful streaming, output configurations, and job modifications to ensure performance and fault tolerance.
Managing the Complete Machine Learning Lifecycle with MLflow - Introduction to MLflow Tracking
Introduction to MLflow: tracking experiments, packaging projects, deploying models, and managing lifecycle. Learn to record, reproduce, and compare ML runs using MLflow's components and UI.
State of the Industry - Machine Learning Platforms and MLOps
Explore the state of MLOps and ML platforms with industry leaders, focusing on challenges, common components, and Databricks' approach to unified data analytics and machine learning.
Accelerating Apache Spark with GPUs Using RAPIDS
Explore GPU acceleration for data science with RAPIDS, offering significant performance gains over CPU-only solutions. Learn its functionality, architecture, and integration with Apache Spark for large-scale computation.
Augmenting Machine Learning with Databricks Labs AutoML Toolkit
Explore how Databricks Labs AutoML Toolkit simplifies and optimizes machine learning processes, from data preparation to model optimization, using financial loan risk data examples.
Simplify and Scale Data Engineering Pipelines with Delta Lake
Learn to build scalable data engineering pipelines using Delta Lake's multi-hop architecture. Discover how to progress from raw data ingestion to machine learning-ready datasets efficiently and effectively.
Introduction to Databricks Lakehouse Platform Architecture and Security
Explore Databricks Lakehouse Platform's architecture, focusing on data reliability, Delta Lake features, and Photon's performance enhancements for efficient data management and analytics.
Low-Code Machine Learning on Databricks with AutoML
Discover how to build, train, and deploy ML models using Databricks AutoML and low-code tools, empowering users of all expertise levels to create predictive analytics solutions.
Implementing End-to-End Demand Forecasting with Databricks and MLflow
Discover how to implement a scalable demand forecasting solution using Databricks and MLflow, improving retail efficiency and reducing food waste through accurate production and delivery planning.