Главная
Study mode:
on
1
Welcome! Prabhat's part of the talk
2
Supercomputers at National Energy Research Supercomputing Center (NERSC)
3
Top 10 data analytics problems (link to blog post)
4
Celeste is at top of the list for many reasons
5
Cori Phase 2, Cray XC-40, the supercomputer that we use
6
NERSC big data stack
7
Performance and productivity tradeoff?
8
Celeste team accomplishment
9
The Celest.jl collaboration
10
Project goals. Jeff Regier's part of the talk
11
Examples of astronomical images that we want to analyze
12
Challenge of analyzing faint stars and galaxies
13
Our approach using Bayesian inference
14
The Celest.jl graphical model
15
Scientific color priors
16
The likelihood: p(images|catalog)
17
Tractable and intractable quantities in Bayesian inference
18
Problem with integral over too high dimensional space
19
Variational inference
20
Julia makes implementation of complicated functions possible
21
Results for small data
22
Validation of our methods against well-researched sky region Stripe 82
23
Numerical optimization scheme and making it parallel
24
Cori Phase 2 supercomputer, some technical information
25
Performance results of Celest.jl
26
Solving large-scale I/O problem. Keno Fischer part of the talk
27
Problem that we encounter with I/O
28
We take down the network with data traffic that we created
29
Standard high-performance guidelines Julia code
30
Majority of hot code in Celest.jl compute Hessian matrix of the objective function
31
Analyzing for loops in the code
32
Question: what is ILP (Instruction Level Parallelism)
33
How much ILP is required?
34
Putting a large chunk of Julia and assemble code in one big scalar inner loop and vectoring it
35
Improving Julia and LLVM to allow the aforementioned vectorization
36
Improvements from the previous point are slowly moving into Julia proper
37
Using StaticArrays.jl
38
Optimization of memory layout to avoid vector shuffling
39
Using multiple dispatch to utilize Hessian matrix structure
40
Optimization takeaways
41
Conclusions
42
Closing information
Description:
Explore petascale computing in Julia through this JuliaCon 2017 conference talk on Celeste.jl. Dive into the world of supercomputers at the National Energy Research Supercomputing Center (NERSC) and learn about top data analytics problems. Discover how Celeste.jl tackles astronomical image analysis using Bayesian inference and variational methods. Follow the speakers as they discuss project goals, challenges in analyzing faint stars and galaxies, and the implementation of complex functions in Julia. Gain insights into performance optimization techniques, including vectorization, memory layout improvements, and utilization of multiple dispatch. Understand how the team solved large-scale I/O problems and achieved impressive performance results on the Cori Phase 2 supercomputer. This comprehensive talk covers everything from the basics of the Celeste.jl project to advanced optimization strategies, making it valuable for both Julia enthusiasts and those interested in high-performance computing in astronomy. Read more

Celeste.jl - Petascale Computing in Julia

The Julia Programming Language
Add to list
#Programming #Programming Languages #Julia #Science #Astronomy #Mathematics #Statistics & Probability #Bayesian Statistics #Bayesian Inference #Computer Science #High Performance Computing #Supercomputing
00:00
-00:26
1 of 42

1 Welcome! Prabhat's part of the talk