Rumour about IBM benchmarks

[John D. McCalpin]

>>>>> On 13 Sep 90 21:18:01 GMT, david at torsqnt.UUCP (David Haynes) said:

David> I have heard a rumour that the benchmark results that IBM posted for
David> their RS6000 system were the results of hand-coded, hand-optimized
David> assembler coding rather than the result of compiling C or FORTRAN
David> code. Can anyone confirm or deny this? 

I can verify that the results of the LINPACK 100x100 test case are
quite valid.  I have not re-run the 100x100 case specifically, but I
have run numerous cases with larger matrices, and have never gotten a
speed as slow as the IBM's rating!  With one LINPACK-ish code (a
block-mode dense linear algebra solver) I get over 13 MFLOPS for
64-bit arithmetic on a model 320!  With the plain vanilla LINPACK
source, I get about 8.4 MFLOPS for the 1000x1000 LINPACK test case.

The MIPS rating of the RS6000's is of course entirely bogus, as it is
based on the Dhrystone 1.0 (or 1.1?) benchmark.  This benchmark is
very easy to optimize for, both in hardware and software, and in my
experience has essentially no correlation with real performance on
non-floating-point work.  For example, both the Motorola 88000 and IBM
RS6000 score substantially higher than the MIPS R-3000 on the
Dhrystone 1.1 benchmark, but the three machines have very similar
scores on the 4 integer SPEC benchmarks. (I do not consider
differences of +/- 25% or less as significant).

On the SPEC benchmarks, the IBM RS6000 model 320 has a floating-point
mean performance of about 24 times a VAX 11/780, and an integer
performance of about 14 times the VAX 11/780.  This level of
floating-point performance is incredible, while the integer
performance is not significantly different than is available in
machines like the DECstation 3100 and DECstation 5000 or in the SUN
Sparcstations. 
--
John D. McCalpin			mccalpin at perelandra.cms.udel.edu
Assistant Professor			mccalpin at vax1.udel.edu
College of Marine Studies, U. Del.	J.MCCALPIN/OMNET