RISC vs CISC simple load benchmark ; amazing !

[Louis Faraut]

Hi Unix Wizards,

here is a posting from France !
*First, excuse me for bad English, I'm *NOT* an English wizard !

We are at present in the process of changing ours school old VAX8650
to a more powerful and less expensive machine :-)

Therefore, we are looking at new RISC technology and how big CPU
manufacturers announced performances are is a matter of wondering for
me :-?

Nevertheless, I'm a bit circumspect ...

So I decided to write a little benchmark program and I ran it on
some RISC machines . 
In this posting, I'd like to submit you some recent and
somewhat surprising results I've got from it :-(

Purpose of my bench was to establish a RISC vs CISC comparison under
heavy CPU loading . For all that, I wrote a general program called
"multiuser" which simulates heavy loading by running many simultaneous
Unix commands then getting back times measured by "getrusage" and
"gettimeofday" system calls .

I try it with different commands to test CPU, I/O etc ... but for the
sake of brevity, I'll only present here results obtained from a
slightly modified version of the famous Bill Joy's test program, where
I believe that RISC are supposed to be better than CISC .

Here is my version of the Joy's program :
========================================
#!/bin/sh
echo 99k65536vvvvvp8op | dc 
========================================

and following are the results (all times in seconds, ratios computed
by the Spreadsheet Calculator)
------------------------------------------------------------
simultaneous commands #  1     2     4     8     16     32

VAX 8650 (8 mips CISC)     Ultrix 1.2 Joy
                user  0.02  0.02  0.05  0.11   0.21   0.48
              system  0.04  0.08  0.19  0.36   0.77   1.58
                real  0.25  0.33  0.53  1.12   2.31   4.59

DEC 5810 (18.7 mips RISC/MIPS)     Ultrix 3.0 Joy
                user  0.01  0.01  0.08  0.10   0.11   0.46
              system  0.10  0.18  0.27  0.66   1.27   2.33
                real  0.14  0.21  0.39  0.83   1.54   3.07
user/VAX              2.00  2.00  0.63  1.10   1.91   1.04 average=1.45
system/VAX            0.40  0.44  0.70  0.55   0.61   0.68 average=0.56
real/VAX              1.79  1.57  1.36  1.35   1.50   1.50 average=1.51


SUN 4/390 (16 mips RISC/SPARC)    SUN OS 4.0.3 Joy
                user  1.74  3.52  7.04 14.00  28.29  57.08
              system  0.10  0.22  0.41  0.88   1.68   3.76
                real  1.93  3.78  7.51 14.99  30.16  61.37
user/VAX              0.01  0.01  0.01  0.01   0.01   0.01 average=0.01
system/VAX            0.40  0.36  0.46  0.41   0.46   0.42 average=0.42
real/VAX              0.13  0.09  0.07  0.07   0.08   0.07 average=0.09
------------------------------------------------------------

DEC5810 exhibits a 1.5* VAX power -not so good indeed !- but main
surprise comes from SUN SPARCServer 4/390 that features for this test
1/10TH THE VAX 8650 PERFORMANCE while rated at TWICE THE POWER !

I'll take care not to conclude that SPARC is bad, because this simple
test does not pretend to check all the aspects of CPU performance
measurement, but as I strongly rely that there is no gross bug in the
source, I think one should be prudent before coming to a decision
because one says : "There's no smoke without fire" !

Would you please give me some feedback about this benchmark ?



Interested in getting multiuser program source ? Send me an electronic
mail . If there are enough requests for it, I'll post it to the net .

*Disclaimer : I'm not DEC remunerated .

                           @
                          ,  ,,     ,,_._.
        	         /  //     //          Jean-Louis Faraut
        	        /  //     //--         
                  //   /  //     //            Administrateur Systeme
                 ((_._'  ((_._. //                   de l'ESSI
        	        	 
E-mail :             +-----------------------------------------------------+
jlf at cerisi.cerisi.fr | ESSI (Ecole Superieure des Sciences Informatiques)  |
jlf at mirsa.inria.fr   |               Sophia-Antipolis (France)             |
Tel. : 93 95 44 37   +-----------------------------------------------------+