Главная
Study mode:
on
1
Intro
2
TOP500: June 2022 Frontier at Oak Ridge AMD EPYC 64C 2GHz, AMD Radeon Instinct GPU. 8,730,112 cores
3
Growth of Problem Size in TOP500
4
Today's Floating-Point Arithmetics
5
Backward Error Analysis for LU Factorizatia
6
Low Precision in Deep Learning
7
The (Partial) Explanation
8
Blocked Inner Products: 2 Pieces
9
Block Summation
10
FABsum Error Bound
11
Random Uniform (0, 1), b = 128, fp32
12
Extended Precision Registers
13
Mixed Precision Block FMA
14
Block FMA Hardware
15
Error Analysis of Block FMAS Blanchard, H, Lopez, Mary, & Pranesh (2020). Analysis of algs for matrix mult c - AB based on block
16
NVIDIA V100
17
Probabilistic Error Analysis Rounding error bounds above are worst-case.
18
Statistical Effects
19
Standard Tool for Rounding Error Analysis Theorem If | sufori - 1 n and nu 1 then
20
Assumptions for Probabilistic Analysis
21
Probabilistic Analysis Theorem (Connolly, H & Mary, 2021)
22
Linear Systems
23
Real-Life Matrices
24
Probabilistic QR Error Bound Theorem (Connolly & H, 2022) Under Model Mand a technical assumption, for the
25
Stochastic Rounding Forsythe (1950). .... Croci et al. (2022).
26
Stagnation Harmonic sum 1/k in fp16.
27
Random Data
28
Putting It All Together
29
Conclusions
Description:
Explore the challenges and solutions in numerical stability of algorithms at extreme scale and low precisions in this 45-minute lecture by Nicholas J. Higham. Delve into the evolving landscape of computer architectures and the approach to exascale computing, examining the implications of using different floating-point precision formats. Investigate how to maximize accuracy in large-scale matrix computations, and learn about innovative techniques to overcome traditional rounding error bounds. Discover the power of blocked algorithms, extended precision registers, and fused multiply-add operations in reducing error constants. Gain insights into probabilistic rounding error analysis and its potential to provide more optimistic error estimates. Examine real-world applications, including deep learning and linear systems, and understand the impact of stochastic rounding on numerical stability. This comprehensive lecture covers topics from TOP500 supercomputers to the latest developments in error analysis, providing a thorough understanding of numerical stability in modern computing environments. Read more

Numerical Stability of Algorithms at Extreme Scale and Low Precisions

International Mathematical Union
Add to list
0:00 / 0:00