Deep learning is supervised learning of parameterised functions by gradient descent
5
Tensor multiplication and non-linearity
6
Algorithms for calculating gradients
7
Composition of Derivatives
8
Mathematician's approach
9
Symbolic differentiation
10
Programmer's approach
11
Automatic differentiation approach
12
Calculate with dual numbers
13
Forward-mode scales in the size of the input dimension
14
Chain rule doesn't care about order
15
Tensor dimensions must agree
16
Solution: expressive type systems
17
Need compilation (to GPU) for performance
Description:
Explore differentiable functional programming in this Scala Days Berlin 2018 conference talk. Dive into parameterised functions, supervised learning, and gradient descent techniques. Understand deep learning as supervised learning of parameterised functions through gradient descent. Examine tensor multiplication, non-linearity, and algorithms for calculating gradients. Compare mathematician's and programmer's approaches to differentiation, including symbolic differentiation and automatic differentiation. Learn about dual numbers, forward-mode scaling, and the chain rule. Discover the importance of expressive type systems and compilation for GPU performance in implementing these concepts.