Are the floating point routines on my machine broken?

[Herman Schuurman]

> .... Is roundup considered acceptable
> behavior for floating point routines?  Or am I somehow misinterpreting
> the problem?
> 
> Ben Broder

Your floating point routines seem to be using the standard IEEE double
precision floating point format, with the rounding mode set to
"round to nearest".  This means that they are rounding to the nearest
representable result, which is quite different from rounding up all the
time.  The accumulated rounding error shows up in the final bit of
the result, which makes it 3.000000000000004e0, rather than 3.0.

The big problem is that .2 is not exactly representable in a binary
floating point format.  It therefore always suffers from rounding
errors.  If you printed the results of your example more precisely,
you would come up with the following results:

0.0e0
0.2e0
0.4e0
0.6000000000000001e0
0.8e0
1.0e0
1.2e0
1.4e0
1.5999999999999999e0
1.7999999999999999e0
1.9999999999999999e0
2.1999999999999999e0
2.4e0
2.6e0
2.8000000000000003e0
3.0000000000000004e0

This clearly shows the rounding errors during addition.  Notice that
some of the results are too low, whereas other results are to high,
indicating a "round to nearest", rather than a "round to positive infinity"
(as IEEE calls rounding up).


-- 
Herman Schuurman 	ARPA:  herman%TI-CSL at CSNET-RELAY.ARPA
Texas Instruments Inc.	CSNET: herman at TI-CSL
PO Box 226015 M/S 238	USENET: {ut-sally,convex!smu,texsun,rice}!ti-csl!herman
Dallas, Texas 75266	VOICE: (214) 995-0845