Statistical physics and statistical inference Lecture 2
6
What is inference?
7
Bayesian inference
8
Efficient codes : parity checks LDPC codes
9
Error decoding: crystal hunting inference problem
10
Error decoding: inference problem
11
Phase Transitions in Error correcting codes
12
Error correction: decoding
13
Phase transitions in decoding
14
Statistical inference: general scheme
15
Bayesian inference with many unknown and many measurements
16
Spin glasses
17
Phase transition with many states: spin glasses
18
Inference, spin- glass and crystal: tomography of binary mixtures
19
Tomography of binary mixtures
20
Crystal : much more probable
21
Inference with many unknowns crystal hunting with mean-field based algorithms
22
Historical development of mean field equations
23
BP = Bethe-Peierls = Belief Propagation
24
BP equations
25
Infinite range models :
26
Example: SK model
27
SK model, TAP equations
28
Two important developments
29
2 What happens in a glass phase, when there are many pure states, and therefore many solutions?
30
SP=BP 2
31
Power of message passing algorithms
32
An example of fully connected model: Generalized Linear Regression
33
Perceptron learning
34
Compressed sensing
35
Spin glass with multispin interactions, infinite range
36
BP equations
37
TAP equations
38
Benchmark: noiseless limit of compressed sensing with iid measurments
39
Analysis of random instances : phase transitions
40
Design the matrix so that one nucleates the naive state crystal nucleation idea,
41
Getting around the glass trap
42
Nucleation and seeding
43
Structured measurement matrix.Variances of the matrix elements
44
Numerical study
45
Performance of AMP with Gauss-Bernoulli prior: phase diagram
46
Many glass states
47
Q&A
Description:
Explore statistical physics and statistical inference in this comprehensive lecture by Marc Mézard, Director of Ecole normale supérieure - PSL University. Delve into Bayesian inference, error-correcting codes, and phase transitions in spin glasses. Examine crystal hunting algorithms, belief propagation, and mean-field equations. Investigate applications in compressed sensing, perceptron learning, and generalized linear regression. Learn about message passing algorithms, TAP equations, and strategies for overcoming glass traps in optimization problems. Gain insights into the connections between statistical physics and machine learning through this in-depth presentation, which includes a Q&A session.
Statistical Physics and Statistical Inference - Lecture 2