NFS for IRIS 3030

[Phil Dykstra]

At BRL we run NFS from the Iris 3030's and 4D's, to both Gould PN
machines and Alliants.  If general it works quite well, and I would
say that its benefits outweigh any of its problems.

For the record, the problems that I know about are:

1) Occasional unreliable data transfer.

This problem is true of the Suns as well (at least I can speak for
SunOS 3.4).  Particularly during very large file copies, say several
megabytes, the new copy may be corrupted.  I have examined three
cases in which this happened, and in every case a single byte had
changed to all ones (0xff).  The location of the bytes in the file
was different every time.

The cause of the corrupted bytes may be software or hardware, I
don't know which, but there is something you can do that will help.
Sun, and thus SGI as well, has tried to increase the efficiency
of NFS by turning off UDP checksumming on incomming and outgoing
NFS/UDP packets.  You can enable checksumming by:

a) Use adb and set "udpcksum" in the kernel to 1.
   This will enable UDP checksumming on *incomming* packets.
   Fix it in your source code too if you've got it.  The file
   is netinet/udp_usrreq.c.

b) For outgoing NFS packets, the routine ku_sendto_mbuf() in
   rpc/subr_kudp.c uses ku_fastsend() which skips checksumming
   rather than the usual udp_output() which does it (if udpcksum
   is nonzero).  You can recompile subr_kudp.c with -DSLOWSEND
   defined which will cause it to use udp_output() instead.

   Unfortunately we don't have source code for the current SGI
   software, so I haven't been able to fix this one yet.  I did
   add this fix to the Sun's, and have caught a few bad packets
   since then that otherwise would have gotten through.

   A pat on the back by the way to whoever at Gould decided NOT
   to use the ku_fastsend() routine.  They don't have this problem.

2) Occasional unreadable files.

This one is real elusive and usually strikes on .rhost files, though
it has happened to other files as well.  I have observed it on the SGI's,
as well as a couple of rare instances on SunOS 3.x (perhaps 3.2 or 3.4,
I'm not sure).

A description I sent a few months ago:
Running with an NFS mounted home directory, rloging into or rcping to a
3030 or 4D, every once in a while the .rhosts file vanishes. The file
entry is still there, and ls even shows its original (correct) length, but
if you try to cat it nothing comes out.  Examining more closely you find
that the open() on the file works, but the first read() fails with
"permission denied."  A local "stat" of the file shows the correct owner
and permissions though.  It is as if either the client or server forgot
who you are (uid wise).  Often touching the file on the server machine,
or perhaps copying it, is enough to make it come back.

If anyone knows what causes this, and/or a fix, I would really appreciate
hearing about it.  Its one of those things that you can never get to happen
when you want it to, so its very hard to debug.

3) Pwd problems (prior to release something-or-other).

The above two problem were generic to Sun's NFS, this one is SGI specific
and has been fixed on the 4D's and patched (mostly anyway) on recent 3030
releases.

Description:
cd to somewhere on an NFS filesystem.  pwd will sometimes die with
"can't change back to ." or "read error in .." or something along those
lines.

The cause of this was 16bit inode numbers (ino_t) on the SGI Irises.
4.nBSD machines such as Suns and Goulds, use 32bit inode numbers.  SGI
had to make the stat() call return a puny inode number.  Truncation from
32 to 16 bits would sometimes map multiple files onto the same dev/inode
pair and thus confuse things like getwd().

On the 4D's SGI went to 32bit inodes and thus don't have this problem.
On the 3030's, ino_t is still 16bits, but stat returns a long inode
instead.  They claim that whatever they did works 99.9+% of the time
and I don't recall seeing it fail since then.  So be it.

- Phil