Part I: Why Good Computations Go Bad

Simply copying formulas out of a math or statistics textbook to plug into a program will almost certainly lead to wrong results. The first part of this book covers the pitfalls of basic numerical computation.

Chapter 1 discusses floating-point numbers in general and how they're different from the real numbers of mathematics. Not understanding these differences, such as the occurrence of roundoff errors, and not obeying some basic laws of algebra can lead to computations that go bad.

Chapter 2 looks at the seemingly benign integer types. They don't behave entirely as the whole numbers of mathematics do. Arithmetic operations such as addition, subtraction, and multiplication take place not on a number line, but on a clock face.

Finally, in Chapter 3, we look at how Java implements its floating-point types. We examine the IEEE 754 floating-point standard and see how well Java meets its provisions.

	Java Number Cruncher: The Java Programmer's Guide to Numerical Computing By Ronald  Mak
	Table of Contents
	Chapter  1.   Floating-Point Numbers Are Not Real!

Consider the common fractions , , , , , and . In the decimal, or base 10, number system, we can represent , , and exactly as 0.5, 0.25, and 0.2, respectively. In the decimal system, we can represent any fraction whose denominator divides evenly into a power of 10 exactly as a decimal fraction. But the denominator 3 doesn't divide evenly, and so repeats infinitely: 0.3333 . . . . Neither does the denominator 6, and is 0.16666 . . . . Neither does the denominator 7, and is 0.142857142857 . . ., where the last group of six digits repeats infinitely.

Like most modern computers, the Java virtual machine uses the binary, or base 2, number system. How well can we represent these fractions in base 2? Program 1-1 prints and sums some of their values. See Listing 1-1.

Listing 1-1 Fractions.

package numbercruncher.program1_1;

/**
 * PROGRAM 1-1: Fractions
 *
 * Print and sum the values of the fractions 1/2, 1/3, 1/4, and 1/5
 * to look for any roundoff errors.
 */
public class Fractions
{
    private static final float HALF    = 1/2f;
    private static final float THIRD   = 1/3f;
    private static final float QUARTER = 1/4f;
    private static final float FIFTH   = 1/5f;
    private static final float SIXTH   = 1/6f;
    private static final float SEVENTH = 1/7f;

    private static final int FACTOR = 840;

    public static void main(String args[])
    {
        System.out.println("1/2 = " + HALF);
        System.out.println("1/3 = " + THIRD);
        System.out.println("1/4 = " + QUARTER);
        System.out.println("1/5 = " + FIFTH);
        System.out.println("1/6 = " + SIXTH);
        System.out.println("1/7 = " + SEVENTH);

        float sum = 0;
        System.out.println();

        for (int i = 0; i < FACTOR; ++i) sum += HALF;
        System.out.println("1/2 summed " + FACTOR + " times = " + sum +
                           " (should be " + FACTOR/2 + ")");

        sum = 0;
        for (int i = 0; i < FACTOR; ++i) sum += THIRD;
        System.out.println("1/3 summed " + FACTOR + " times = " + sum +
                           " (should be " + FACTOR/3 + ")");

        sum = 0;
        for (int i = 0; i < FACTOR; ++i) sum += QUARTER;
        System.out.println("1/4 summed " + FACTOR + " times = " + sum +
                           " (should be " + FACTOR/4 + ")");

        sum = 0;
        for (int i = 0; i < FACTOR; ++i) sum += FIFTH;
        System.out.println("1/5 summed " + FACTOR + " times = " + sum +
                           " (should be " + FACTOR/5 + ")");

        sum = 0;
        for (int i = 0; i < FACTOR; ++i) sum += SIXTH;
        System.out.println("1/6 summed " + FACTOR + " times = " + sum +
                           " (should be " + FACTOR/6 + ")");

        sum = 0;
        for (int i = 0; i < FACTOR; ++i) sum += SEVENTH;
        System.out.println("1/7 summed " + FACTOR + " times = " + sum +
                           " (should be " + FACTOR/7 + ")");
    }
}

Output:

1/2 = 0.5
1/3 = 0.33333334
1/4 = 0.25
1/5 = 0.2
1/6 = 0.16666667
1/7 = 0.14285715

1/2 summed 840 times = 420.0 (should be 420)
1/3 summed 840 times = 279.99915 (should be 280)
1/4 summed 840 times = 210.0 (should be 210)
1/5 summed 840 times = 167.99858 (should be 168)
1/6 summed 840 times = 139.99957 (should be 140)
1/7 summed 840 times = 120.001114 (should be 120)

We appended an f to some of the numeric literals in the program to make them single-precision float numbers. That way, 1/2f uses floating-point instead of integer arithmetic.

The first set of the program's output lines doesn't look too bad; , , and appear to be fine. There's a small roundoff error for C it's a bit odd that the rightmost digit got rounded up, but the error is quite small. There are similarly small roundoff errors for and .

The second set of the output lines shows what happens when we let even small errors accumulate. Although there was a rounding up error in , we now see that it accumulated a rounding down error! Evidently, there was initially a tiny hidden error in that accumulated a rounding down error. also accumulated a rounding down error, and accumulated a rounding up error. and apparently had no errors, but then, of course, they are exact powers of 2: and .