Omniadisce

Applied maths 14.5: Gradient descent and Newton's method

Lesson Advanced

Join free to take this lesson

Anyone can look around, but taking a lesson needs a free account. With one, your answers are checked step by step and your progress is saved, so you can stop and pick up where you left off on any device.

10 steps ยท about 17 min

By the end you can

  • Carry out a step of gradient descent $x_{n+1} = x_n - h\,f'(x_n)$, in one and two variables
  • Explain the effect of the step size: too small is slow, too large diverges
  • Carry out a step of Newton's method for minimisation, $x_{n+1} = x_n - f'(x_n)/f''(x_n)$

Who it is for: You know the gradient (lesson 4.2), Newton's method for equations (lesson 2.5) and convex functions (lesson 14.4).

The steps

  1. Which way is downhill?
  2. Gradient descent
  3. One step
  4. The step size matters
  5. Too big a step
  6. A step in two variables
  7. Newton's method for minimisation
  8. One Newton step
  9. Which method for a big model?
  10. Choosing the step size

Discussion 0 comments

Log in to join the discussion.

× Enlarged view