Главная
Study mode:
on
1
Intro
2
What are we going to cover today?
3
Data Pipelines Crash Course
4
Observability
5
Metrics Collection
6
Pipeline Latency
7
Status Monitoring
8
Immutable Inputs
9
Benefits of Idempotency and immutability
10
Declarative Pipeline Example
11
Benefits of Pipelines as Code
12
Usable Data
13
Pros and cons of Data Validation
14
Adding Validators to Data Pipelines
15
Common Concerns
Description:
Explore strategies for designing resilient data pipelines in this 30-minute conference talk from SREcon19 Americas. Learn how to quickly recover from failures, ensure data accuracy, and minimize risks associated with pipeline updates. Discover techniques for reducing operational complexity, implementing observability, and enhancing data pipeline reliability. Gain insights into metrics collection, status monitoring, immutable inputs, declarative pipelines, and data validation. Address common concerns and best practices for maintaining robust data pipelines in complex environments.

Designing Resilient Data Pipelines

USENIX
Add to list