Главная
Study mode:
on
1
Introduction
2
Creating The Spark Cluster and Airflow on Docker
3
Creating Spark Job with Python
4
Creating Spark Job with Scala
5
Building and Compiling Scala Jobs
6
Creating Spark Job with Java
7
Building and Compiling Java Jobs
8
Cluster computation results
Description:
Learn to create and submit PySpark jobs to Spark clusters in this comprehensive tutorial. Dive into an end-to-end data engineering project combining Apache Airflow, Docker, Spark Clusters, Scala, Python, and Java. Create basic jobs using multiple programming languages, submit them to the Spark cluster for processing, and observe live results. Follow along as the instructor guides you through setting up a Spark cluster and Airflow on Docker, creating Spark jobs in Python, Scala, and Java, building and compiling Scala and Java jobs, and analyzing cluster computation results. Gain practical experience in big data processing and workflow automation, essential skills for aspiring data engineers.

Creating and Submitting PySpark Jobs to Spark Clusters

CodeWithYu
Add to list