rlogins hanging under 4.0

[Christian H. Debrunner]

In v7n139 Danielle Heinzer <ESC1298 at ESOC.BITNET> writes:

>Since one week, I have observed the following: I cannot rlogin the Sun
>3/260 file server in a shelltool from my Sun3/50. I see the motd and the
>prompter of the remote file server, but I cannot enter anything....

I have also had problems with rlogins, cmdtools, and shelltools hanging,
and I found that it only occurred when the pseudo-tty (pty) assigned to
the rlogin, cmdtool or shelltool was open for input or output in some
other (usually a background) process.

I saw two different failure modes.  The rlogin, cmdtool, or shelltool
either crashed when the first character was typed (it seemed to interpret
every character as a ^D), or it would seem to be in "raw" mode (no echo,
no CR -> NL mapping).  I sent a detailed report of how to recreate this
bug to the SUN hotline, but they were unable to reproduce it.  After I had
sent them several test cases that demonstrated the bug on our system, none
of which they could reproduce, they wrote to say they were "researching
the problem".  That was in November.

I have found two solutions (workarounds) to this problem:

1)  Stop (~ ^Z) the rlogin or close (but do not quit) the broken
    cmdtool or shelltool.  This will tie up the broken pseudo-tty (pty), so
    the next rlogin, cmdtool, or shelltool will use a different pty.  Repeat
    until you get a working one.  Note that if you kill the rlogin, cmdtool,
    or shelltool and start a new one, you will most likely get the same pty
    again and it will still not work.

2)  Find out which pty the rlogin, cmdtool or shelltool is using, and use
    that handiest of handy utilities "ofiles" (available in the archives) to
    find all processes which have that pty open.  If the processes in question
    are expendable (or easily restarted), kill them.  The problem can be
    avoided by running all background jobs with stdin, stdout and stderr
    redirected.  Since our users have started running their background
    jobs with all IO redirected, we have not seen any more hung ptys.

I realize that there are any number of other possible causes for the
symptoms described in the original note, but when I read that a shelltool
once hadn't worked on the server, it occurred to me that they might be
experiencing the hanging pty problem.

	Chris Debrunner
	debrunne at uicsl.csl.uiuc.edu
	(217) 333-1368