Omniadisce

Applied maths 15.6: Neural networks and gradient descent

Lesson Advanced

Join free to take this lesson

Anyone can look around, but taking a lesson needs a free account. With one, your answers are checked step by step and your progress is saved, so you can stop and pick up where you left off on any device.

10 steps ยท about 18 min

By the end you can

  • Carry out a gradient-descent step $\theta \leftarrow \theta - \eta L'(\theta)$
  • Find how large the step size can be before gradient descent diverges
  • Compute a small network's output and count its parameters

Who it is for: You know derivatives and the chain rule (lessons 2.2-2.4) and partial derivatives (lesson 4.2).

The steps

  1. Too big a step
  2. Gradient descent
  3. One step
  4. Fitting a weight
  5. The largest safe step
  6. A neural network
  7. Run a tiny network
  8. Count the parameters
  9. Reading a loss curve
  10. Why the chain rule matters

Discussion 0 comments

Log in to join the discussion.

× Enlarged view