RMS deviation between two sets of cartesian coords

James F. Blake jim at doctor.chem.yale.edu
Fri Dec 7 07:46:50 AEST 1990


>From article <14666 at smoke.brl.mil>, by gwyn at smoke.brl.mil (Doug Gwyn):
> 
> It's hard to see how you could get 5% fuzz in a variance computation
> if you use the direct definition (sums of squares of deviations).
> The textbook rewrite as mean-of-squares minus square-of-mean is well
> known to produce bogus answers, however, particularly when the variance
> is relatively small.

     You're correct.  I believe the problem was a combination of the
equations and a "feature."  The original code allowed the axis to be 
transformed such that they were nonorthogonal, if this would produce a
better fit.  This probably produced most of the fuzz.  Thanks to all who
have responded, your insight is very much appreciated.

  Jim



More information about the Comp.lang.c mailing list