Omniadisce

Applied maths 10.1: Floating point, rounding and error

Lesson Intermediate

Join free to take this lesson

Anyone can look around, but taking a lesson needs a free account. With one, your answers are checked step by step and your progress is saved, so you can stop and pick up where you left off on any device.

10 steps ยท about 16 min

By the end you can

  • Explain why a computer cannot store most decimals exactly, and what machine epsilon means
  • Compute absolute and relative errors, and turn a relative error into significant digits
  • Spot subtraction of nearly equal numbers, and rewrite a formula to avoid it
  • Tell round-off error from truncation error, and find the step size that balances them

Who it is for: You have finished units 1-9. You need a calculator, and some ease with powers of 10.

The steps

  1. Guess first
  2. Numbers in binary
  3. Measuring error
  4. Relative error
  5. Subtracting nearly equal numbers
  6. Which formula is safer?
  7. Two kinds of error
  8. The best step
  9. The Patriot clock
  10. Sixteen digits?

Discussion 0 comments

Log in to join the discussion.

× Enlarged view