Join free to take this lesson
Anyone can look around, but taking a lesson needs a free account. With one, your answers are checked step by step and your progress is saved, so you can stop and pick up where you left off on any device.
10 steps ยท about 16 min
By the end you can
- Explain why a computer cannot store most decimals exactly, and what machine epsilon means
- Compute absolute and relative errors, and turn a relative error into significant digits
- Spot subtraction of nearly equal numbers, and rewrite a formula to avoid it
- Tell round-off error from truncation error, and find the step size that balances them
Who it is for: You have finished units 1-9. You need a calculator, and some ease with powers of 10.
The steps
- Guess first
- Numbers in binary
- Measuring error
- Relative error
- Subtracting nearly equal numbers
- Which formula is safer?
- Two kinds of error
- The best step
- The Patriot clock
- Sixteen digits?
Discussion 0 comments
Log in to join the discussion.