layers hangs terminal

[~XT6510300~Frank McGee~C23~L25~6326~]

In article <676 at msa3b.UUCP> kevin at msa3b.UUCP (Kevin P. Kleinfelter) writes:
>I'm running layers on an AT&T 3B2, with Sys V R3.2.1, and an AT&T 630 tty.
>I'm pretty pleased, but there is this problem...
>If I accidentally do the following
>	login successfully
>	start layers
>	do inconsequential stuff
>	^D (without leaving layers)
>The layer is "dead".  Since it is my only layer, the terminal is dead.
>After killing off all the "xt" entries and "layers" and "ksh" that I can
>find via "ps -e" from another terminal, the 630 is still dead.  Turning the
>630 off and on does not help.  It is not available again until the system is
>rebooted.
>Two questions:
>	1) How can I get my terminal back without a reboot?
>	2) How can I tell (in a shell script) if layers is active on my tty?
>	   (If I can set my prompt string, PS1 to say "LAYERS" I can
>	    avoid trying to logout while layers is running.)
>Kevin Kleinfelter @ Management Science America, Inc (404) 239-2347
>gatech!nanovx!msa3b!kevin

There are two issues here; the layers software and the cable used to
connect the terminal to the host.

If you type a ctrl-d in a shell running in a layers window the shell
will end, but layers won't exit until you tell it to.  You should still
be able to use the mouse, and pull down the 630 menu and there should
be an option to quit layers.  If you quit layers, you will have your
original login shell back (you will still be logged into the host) and
you can either logout from there or start layers again.  In addition I
believe there is an option from within the 630 menus to disconnect from
the host.  This won't quit layers on the host for you, but it will
disconnect you.

It also sounds like you aren't using a fully-wired cable to connect to
your host either.  If you are using a fully wired cabled, your serial
port on the host should be able to sense when you power down the 630,
and it should kill off all processes associated with that tty when it
senses that DTR has dropped.  If you are using a 3-wire serial cable
that's probably the reason why layers is hanging the port.  A fully
wired cable looks something like this (there are several variations) :

	host	630
	1     --  1
	2     --  3
	3     --  2
	4     --  5
	5     --  4
	6 & 8 -- 20
	7     --  7
	20    --  6 & 8

Basically pins 1 and 7 are straight through, you swap pins 2 and 3, you
swap pins 4 and 5, and tie 6 and 8 on each side to pin 20 on the other
side.   This cable will allow the serial port on your host to sense
that DTR has dropped and things will behave normally.  Some other
things to check would be /etc/gettydefs (make sure HUPCL is set),
/etc/inittab (make sure the tty you are coming in on is a hardware flow
control port), and make sure you aren't trapping signal 1 (SIGHUP) in
your .profile or /etc/profile.

I don't know how you could tell that you were running under layers from
within a shell script; you might want to try having another script that
sets an environment variable, exports it, then starts layers.
Something like :

LAYERS=YES
export LAYERS
layers

Hope this helps, good luck.

-- 
Frank McGee, AT&T
Tier 3 Indirect Channel Sales Support
attmail!fmcgee