Questions on network transfer rates

[Chris Jankowski]

In article <1991May1.233042.7715 at ims.alaska.edu> ddr at flux.isr.alaska.edu (Donald D Rice) writes:
>
>I'm trying to find some practical estimates of how fast data can be moved over
>a tcp/ip ethernet (10 Mbps).  I ran some tests with two DS5000/Ultrix 4.0
>machines connected only to each other with no other users to get an idea
>of how fast they could transfer data.  For a test file, I used a 10.8 MB
>data file.  The results I got were:
>
>	NFS cp:		410 kB/s
>	FTP:		395 kB/s
>	TFTP:		178 kB/s
>
>Questions:
>1) I thought that for a clear 10 Mbps tcp/ip link, the data transfer rate would
>   be on the order of 1 MB/s (data, not counting packet headers and such).  If
>   that is too optimistic, why?  If it is realistic, why are the workstations
>   putting out at less than half of the maximum speed?  In the latter case,
>   connecting FDDI directly to the workstations would seem pointless.

I believe that those are quite good results - by transfering a file you
also work through file systems on *both* workstations so your CPUs have
to serve those requests as well **and** for large enough files you are paced
by disk accesses (once buffer cache is full).

You are then on mercy of an electromechanical device we still have to
use for mass storage.
 
I have seen results from a program sending data generated internally
and discarding it on input (no disk I/O).
For two DEC5000 (Ultrix 4.?) that yields 9.1Mbits per second of data
(the protocol overhead will easily account for the rest) using TCP.
That means that DEC5000 has more than enough CPU cycles to drive
Ethernet full speed.

To roughly gauge the impact of the filesystem accesses you could do
a file transfer on your workstation between 2 filesystems - and remember
about taking into account effects of buffer cache and BSD FFS.

The same program between two DEC5000 with FDDI yields 32Mbits per second
of data and only then DEC5000 CPU works flat out.

So, IMHO FDDI on DEC5000 makes perfect sense in general ie. when it is
used in a multiprocessing enviroment where data may come from many sources
some of them being data areas kept in memory, buffer cache or more than
one disk spindle and controller. 

>2) I was surprised that NFS cp did so well.  Under normal network conditions,
>   cp is always slower than FTP on our machines.  One time when the network was
>   fairly busy, FTP moved the test file at 240 kB/s (60% of maximum rate),
>   while NFS cp moved the file at only 8.1 kB/s (2% of maximum?!).  Is that
>   kind of degradation normal for NFS, or does it suggest a problem with our
>   configuration?

This is normal effect of UDP protocol (no, it is not broken it just may
produce this kind of side effect).

In the test case you had CPUs of mached speed, presumably quiet machines
and network.
Everything self balanced (by chance).
By the way it does matter whether the source or destination are remotely
mounted. Did you try both cases?

I give you a different example.
Take DEC5000 and from it throw UDP packets with maximum speed at
a DEC2100. DEC5000 will report about 9Mbits per second sent but
poor DEC2100 will not let you even to move your mouse pointer and it
will receive nothing so overrun it will be.
Again there is nothing wrong with UDP or DEC2100. They all work as
designed.

The explanation lies in differences between TCP protocol 
(used by FTP) and UDP protocol (used by NFS).
In the nutshell UDP does not send any acknowledgements for received
packets unless the higher layer protocol aranged that.

>3) I was also surprised that TFTP was so slow compared to FTP.  We normally
>   have TFTP disabled, but I turned it on for these tests because I thought
>   its simpler transfer mechanism would give faster transfer rates.  Why
>   not?

TFTP can be run on top of either UDP or TCP.
Which version did you use?

>4) Has anybody else run tests like these for other machines?  I'd be interested
>   to see how different combinations of PCs/software/network cards work out.

A general comment.
With those tests as with all scientific experiments it is extremly important 
to control enviroment in which you test and measure.

Lots of people run those tests. There is quite nice tool in PD you may
wish to use. It is called netbench(1) I believe. It generates data
internally so you can measure networking only.

For fine analysis a good protocol analyzer like Network General SNIFFER
and sound knowledge of relevant protocols are indispensable.

Hope this helps.
 
 +-+-+-+-+-+-+-+  Chris Jankowski - Open Systems Eng.- chris at ubeaut.enet.dec.com
 |d|i|g|i|t|a|l|  Digital Equipment Corporation (Australia)   tel.+61 3 655 5622
 +-+-+-+-+-+-+-+  45 Exibition St, Melbourne 3000, AUSTRALIA  fax +61 3 655 5655

 ... and in the last 50 years ... there has never been a famine in a country 
 with a free press         - Lawrence Summers - chief economist - World Bank