19.6 Accuracy

I l @ ve RuBoard

How many digits of the fraction are accurate? At first glance you might be tempted to say all four digits. Those of you who have read the previous section on roundoff error might be tempted to change your answer to three.

The answer is: the accuracy depends on the calculation. Certain operations, such as subtracting two numbers that are close to each other, generate inexact results. Consider the following equation:

1 - ¹ / ₃ - ¹ / ₃ - ¹ / ₃

In floating-point notation this is:

 1.000E+0 -0.333E-0 -0.333E-0 -0.333E-0 _______________________  0.0010E+0, or 1.000E-3 

The correct answer is 0.000E+0 and we got 1.000E-3. The very first digit of the fraction is wrong. This is an example of the problem called roundoff error that can occur during floating-point operations.

I l @ ve RuBoard