Главная
Study mode:
on
1
Introduction
2
Data Pipelines
3
Use Cases
4
What is ETL
5
Pipeline options
6
Example
7
Lobby Focus
8
Familiar tools
9
Scrape
10
Redis
11
Fanning out
12
Fanning out view
13
Load job
14
Unique index
15
Normalizing
16
Stuff Changes
17
Will this scale
18
Options
19
Opportunities
Description:
Explore strategies for processing large-scale data using Ruby on Rails in this 35-minute conference talk. Learn how to handle massive datasets from various sources, including public records, shared resources, and IoT data. Discover techniques for ingesting, transforming, and storing data efficiently using familiar tools like ActiveRecord, Sidekiq, and Postgres. Gain insights into breaking down large data jobs into manageable tasks, turning data firehoses into structured pipelines. Dive into topics such as ETL processes, pipeline options, and practical examples using lobby data. Understand the importance of scraping, Redis usage, fanning out processes, load jobs, unique indexing, and data normalization. Address scaling concerns and explore opportunities for optimizing data processing in Rails applications.

Processing Data at Scale with Rails

Ruby Central
Add to list
0:00 / 0:00