Floating point non-exactness

[Richard A. O'Keefe]

In article <622 at .tetrauk.UUCP>, rick at .tetrauk.UUCP (Rick Jones) writes:
> 	fpcmp (double a, double b)
> 
> 		returns 0 if a equals b within the reliable precision,
> 		else returns 1 if a > b, or -1 if a < b
> 
> Real problem:  how do you write fpcmp() ?

See Knuth, "The Art of Computer Programming", vol 2, "Semi-Numerical Methods"
He defines "approximate" versions of <, >, and *TWO* approximate versions of
==.   Note that the appropriate "tolerance" to use does NOT depend only on
the properties of your floating-point arithmetic, but on details of your
algorithm.

The ultimate answer to your question is to take a course on numerical
analysis.  I sometimes wonder whether there ought to be a generally
understood qualification for computer programmers:  LFP (licensed to
use floating-point) (:-).

> Can you ensure that "fpcmp (a, b)" and "fpcmp (a-b, 0.0)" yield the same result
> when a and b are very close?

No.  In many computer arithmetic systems, a-b may collapse to a precise 0.
See the axioms Knuth gave.
-- 
Science is all about asking the right questions.  | ok at goanna.cs.rmit.oz.au
I'm afraid you just asked one of the wrong ones.  | (quote from Playfair)