Главная
Study mode:
on
1
Introduction
2
Demo Overview
3
Han Wang Introduction
4
First Example
5
Spark
6
Transformation
7
Fugue Code
8
Model
9
Field Workflow
10
Results
11
Physical
12
Prediction
13
Pandas vs Spark
14
Lazy evaluation of Spark
15
Partitioning
16
Testing
17
Fugue
18
Decouple logic and execution
19
Demo
20
Notebook extension
21
Conclusion
22
Recap
Description:
Explore scaling machine learning workflows to big data using Fugue in this 29-minute conference talk from KubeCon + CloudNativeCon Europe 2022. Learn how to transition from Pandas to distributed computing frameworks like Spark or Dask without reimplementing code. Discover Fugue's open-source abstraction layer that allows data scientists to write framework-agnostic and scale-agnostic code. Follow along as the speakers demonstrate porting native Python code to Spark or Dask with minimal changes, and witness the scaling of data compute from a single machine to a Spark cluster on Kubernetes. Gain insights into lazy evaluation, partitioning, testing, and decoupling logic from execution in big data workflows.

Scaling Machine Learning Workflows to Big Data with Fugue

CNCF [Cloud Native Computing Foundation]
Add to list