Главная
Study mode:
on
1
Introduction
2
What is a data system
3
Common sources of complexity
4
Human fault tolerance
5
Design for human error
6
Data loss
7
Mutability
8
Immutability
9
Normalization vs Denormalization
10
Denormalization
11
Schemas
12
Schemas are bad
13
Schemas are confusing
14
What is a schema
15
What is structural integrity
16
Preventing corruption
17
Detecting corruption
18
Preventing mistakes
19
Learning from experience
20
Why schemas are painful
21
My ideal schema tool
22
Apache Thrift
23
New Sequel
24
No Sequel
25
How would you build a better data system
26
What do we actually use data systems for
27
Data Systems
28
Example
29
Realtime Queries
30
Pre Computation
31
Pre Computation Example
32
Architecture
33
Functions
34
View
35
Batch Processing
36
MapReduce
37
BatchView Databases
38
BatchView Properties
39
BatchView Architecture
40
Batch Computation
41
RealTime Views
42
Lambda Architecture
43
Cap Theorem
44
Eventually Accurate
45
Maximizing Value
46
Tools
47
Land Architecture
48
Movement Mistakes
49
Normalization Personalization
50
The Future
51
Book
52
Performance
Description:
Explore a comprehensive plan to address runaway complexity in big data systems in this conference talk from GOTO Aarhus 2012. Learn how embracing immutability and moving away from the CRUD paradigm can simplify data systems. Discover the role of NoSQL in the big picture and understand the "Lambda Architecture," a generic approach combining batch and real-time processing. Delve into topics such as human fault tolerance, data loss prevention, normalization vs. denormalization, and the challenges of schemas. Gain insights on building better data systems, pre-computation techniques, and the CAP theorem. Examine the architecture of batch views, real-time views, and the overall Lambda Architecture. Conclude with a look at future trends and performance considerations in big data systems.

Runaway Complexity in Big Data Systems and a Plan to Stop It

GOTO Conferences
Add to list
#Conference Talks #GOTO Conferences #Programming #Databases #NoSQL Databases #Data Science #Data Processing #Big Data #Lambda Architecture
1 of 52

Introduction