Calculate the derivative of the loss function with respect to the parameters
6
Calculate gradient for current parameters
7
Deep learning is supervised learning of parameterised functions by gradient descent
8
Tensor multiplication and non-linearity
9
Tensors: multidimensional arrays
10
Conditionals and loops
11
Algorithms for calculating gradients
12
Composition of Derivatives
13
Symbolic differentiation
14
No loops or conditionals
15
Inexact. Choosing e is hard
16
Two main algorithms: forward and reverse
17
Calculate with dual numbers
18
There is a monad for Dual
19
Forward-mode scales in the size of the input dimension
20
Chain rule doesn't care about order
21
Use a monad (or continuations)
22
flatMap is the chain rule
23
Reverse-mode scales in the size of the output dimension
24
Tensor dimensions must agree
25
Solution: expressive type systems
26
Linear logic and differential A-calculus
27
Need compilation (to GPU) for performance
28
Scala is well positioned for the next generation of DL systems
Description:
Explore differentiable functional programming in this Scala Days conference talk. Dive into parameterised functions, supervised learning, and gradient descent techniques. Learn about deep learning as supervised learning of parameterised functions, tensor multiplication, and non-linearity. Discover algorithms for calculating gradients, including symbolic differentiation, forward-mode, and reverse-mode approaches. Examine the use of monads and continuations in gradient calculations, and understand how Scala's expressive type system can address tensor dimension challenges. Gain insights into linear logic, differential λ-calculus, and the potential for Scala in developing next-generation deep learning systems.