A round-off error, also called rounding error, is the difference between the calculated approximation of a number and its exact mathematical value. Numerical analysis specifically tries to estimate this error when using approximation equations and/or algorithms, especially when using finite digits to represent infinite digits of real numbers. This is a form of quantization error.
| Notation | Represent | Approximate | Error |
|---|---|---|---|
| 1/7 | 0.142 857 | 0.142 857 | 0.000 000 142 857 |
| ln 2 | 0.693 147 180 559 945 309 41... | 0.693 147 | 0.000 000 180 559 945 309 41... |
| log10 2 | 0.301 029 995 663 981 195 21... | 0.3010 | 0.000 029 995 663 981 195 21... |
| ∛ 2 | 1.259 921 049 894 873 164 76... | 1.25992 | 0.000 001 049 894 873 164 76... |
| √ 2 | 1.414 213 562 373 095 048 80... | 1.41421 | 0.000 003 562 373 095 048 80... |
| e | 2.718 281 828 459 045 235 36... | 2.718 281 828 459 045 | 0.000 000 000 000 000 235 36... |
| π | 3.141 592 653 589 793 238 46... | 3.141 592 653 589 793 | 0.000 000 000 000 000 238 46... |
Double-rounding can increase the round-off error. For example, if the numeral 9.945309 is rounded to two decimal places (9.95) for data entry purposes, and then rounded again to one decimal place (10.0) for display purposes, the apparent round-off error is 0.054691. If the original number was rounded to one decimal place in one step (9.9), the round-off error is only 0.045309.
There are at least two ways of performing the termination at the limited digit place: