Главная
Study mode:
on
1
Intro
2
About me
3
What is a workflow?
4
Workflows show up all over the place
5
Example: Processing Uploaded Images
6
Workflow is defined by a task graph
7
Workflow system design
8
Workflow for software builds
9
Design goals
10
Task API design challenges
11
Python to the rescue!
12
Building the rule graph
13
Static validation
14
Type annotations provide
15
Why coroutines?
16
Coroutines are powerful
17
Coroutines can express concurrency
18
Coroutines help avoid side effects
19
Coroutines provide natural control points for
20
Summary
Description:
Explore the intricacies of creating extensible workflows using unconventional Python techniques in this PyCon US talk by Benjy Weinberger. Dive into the challenges of designing a programmatic interface for workflow systems, balancing flexibility and constraints. Learn how the Pants build system leverages Python's type annotations, coroutines, and dataclasses to create an elegant extensibility API. Discover how these features can be combined to support complex functionalities like caching, concurrency, and remote execution without burdening task authors. Gain insights into workflow system design, task API challenges, and practical applications in data processing, ETL, and CI/CD. Follow along with real-world examples and understand how to apply these concepts to build more efficient and maintainable workflow-oriented systems.

Creating Extensible Workflows with Off-Label Use of Python

PyCon US
Add to list
0:00 / 0:00