RISC MIPS -- Sun vs. VA

[Richard Wood]

One significant point of contention about the meaning of "MIPS" is the
fact that no commonly used test or benchmark measures only the hardware of
the system.  Each will measure a different blend of raw hardware speed
(which combines many related subsystems) as well as OS performance and
compiler efficiency.

Most of the earlier MIPS ratings cited by vendors in the UNIX market
benchmarked their machines against a VAX 11/780 running 4.1BSD or 4.2BSD.
Since those are both quite a few years old, they aren't really valid
measurements now.  Thus the performance of an 11/780 changes over time -
today's software will deliver more power than yesterday's.

Since any VAX running a contemporary copy of VMS and it's compilers will
out-perform the equivalent hardware running 4.2, there is a distinct bias.
Digital realized this bias was being used as a marketing weapon, and
started using the term "VUPS", for VAX Units of Processing, defined as the
performance of a VAX 11/780 running the most recent VMS software.  The
most recent Ultrix software will perform roughly the same (a little better
in C-based programs, a little worse in Fortran-based - due mostly to the
design intentions of the OS, as opposed to compiler quality).

Digital Review uses MVUPS as the basis for their quite well thought out
suite of tests.

Of course, now most vendors that compete directly with Digital use the
term "VAX MIPS" (or even VUPS) when claiming performance numbers.  Whether
these figures are adjusted for the differences in OS/Compilers can only be
known by that companies marketing people.

The only sure way of testing the performance of any machine is to run the
application mix you intend to use on the machines you are considering.
Failing that, pay more attention to independent tests such as those in
industry magazines.  Vendor claims should be thought of as the propaganda
they usually are.