PySpark with Python
Learn PySpark for big data processing: DataFrame operations, handling missing values, filtering, aggregations, MLlib basics, and Databricks integration. Includes linear regression implementation.
Machine Learning Using Spark MLLib
Learn to implement machine learning algorithms using Spark MLLib, covering linear regression techniques, data preprocessing, model building, and evaluation methods in a hands-on, practical approach.
SQL está en todas partes: Explorando alternativas con Ibis
Descubre Ibis: una biblioteca Python para consultas de datos eficientes en múltiples backends. Simplifica el trabajo con bases de datos SQL y ofrece una alternativa a las complejas consultas tradicionales.
MLOps MLFlow: Mlflow Projects with PySpark Docker Environment and MLflow
Aprende a utilizar MLflow Projects con entornos Pyspark y Docker para gestionar flujos de trabajo de aprendizaje automático de manera eficiente y reproducible.
State of the Art Natural Language Understanding at Scale - David Talby
Explore state-of-the-art NLP techniques using Apache Spark, covering core algorithms, distributed pipelines, and performance optimizations for large-scale language understanding applications.
Accelerating MLOps with Apache Spark and SingleStore
Learn to accelerate MLOps using Apache Spark and SingleStore. Discover how to build fast analytical applications, manage supervised learning models, and scale AI/ML operations for data science success.
Project Zen - Improving Apache Spark for Python Users
Discover improvements in Apache Spark for Python users, including redesigned documentation, type hints, PyPI distribution options, and enhanced PySpark features for a more Pythonic experience.
Deep Dive into New Features of Apache Spark 3.1
Explore Apache Spark 3.1's new features: ANSI SQL compliance, streaming enhancements, Python improvements, and performance optimizations for faster, easier, and smarter data processing.
Comprehensive View on Intervals in Apache Spark 3.2
Explore Apache Spark 3.2's interval enhancements, including ANSI SQL conformance, new interval types, and improved APIs. Learn to construct, use, and manage intervals effectively in Spark SQL and PySpark.
Master Databricks and Apache Spark Step by Step - PySpark Using RDDs
Explore PySpark and RDDs in Databricks: Learn about lazy evaluation, transformations, and actions while analyzing data using Resilient Distributed Datasets.
Data Science Across Data Sources with Apache Arrow - Accelerating Analytics and Interoperability
Explore Apache Arrow's columnar in-memory data representation for real-time analytics across distributed systems. Learn how it accelerates data processing and enables interoperability between various analytical tools and data sources.
PySpark in Apache Spark 3.3 and Beyond
Explore PySpark's evolution in Apache Spark 3.3, including improved Pandas API, enhanced productivity features, and future developments. Learn about new functionalities, usability improvements, and performance enhancements.
Simplifying Testing of Spark Applications
Learn techniques to simplify testing Spark applications, covering Python UDFs, Pandas UDFs, and functional APIs. Explore practical scenarios and demos for efficient data processing and analysis.
Let's Play with Charmed Spark - Apache Spark on Kubernetes
Explore Apache Spark on Kubernetes using Charmed Spark. Learn to deploy, configure, and run Spark workloads, integrate with Kafka, and monitor jobs through hands-on demonstrations.
Data Engineering with Databricks - Leveraging the Lakehouse Platform for ETL Pipelines
Leverage Databricks Lakehouse Platform to build ETL pipelines, use Delta Live Tables, and orchestrate tasks with Workflows for efficient data processing.
AWS EMR Demystified - Part 4
Master AWS EMR's core components, from networking and storage to running PySpark jobs, while exploring integration with AWS Glue and Athena for effective big data processing and transformation.
AWS EMR Demystified - Part 3
Master AWS EMR's integration with Glue, Athena, and PySpark while learning essential data transformation techniques and architectural considerations for optimizing your data lake implementation.
AWS EMR, Hadoop, and Data Lake Architecture - Part 2
Master AWS EMR's core components, including networking, storage types, and integration with AWS Glue and Athena. Learn to execute PySpark jobs and understand data transformation challenges in EMR-based data lakes.
PySpark Best Practices
Learn best practices for PySpark, including dependency management, code structuring, testing, and performance optimization. Gain insights on writing efficient and scalable Python applications using Spark's distributed computing model.
From Python to PySpark and Back Again - Unifying Single-Host and Distributed Deep Learning with Maggy
Unifying single-host and distributed deep learning with Maggy framework, enabling seamless transition from laptop to cluster for efficient model development and training using TensorFlow and PySpark.
Architecting for Data Quality in the Lakehouse with Delta Lake and PySpark
Learn techniques for ensuring data quality in lakehouses using Delta Lake and PySpark. Explore data observability, reliability lifecycle, and practical implementation of quality checks across metadata, storage, and query layers.