pty in 4.1BSD (summary: long)

[David Brown]

In March, I asked the followng question about ptys in 4.1BSD, and
received a good number of replies - thanks to everyone who responded.
I also received a few requests for summaries, so here it is.
In the name of brevity, I have chopped off all headers, etc. I still
have the original replies; if you wish to contact the author of any
of these notes, drop me a line.

	David Brown  (415) 945-2199
	Varian Instruments 2700 Mitchell Dr Walnut Creek CA 94598
	{zehntel,fortune,amd70}!varian!david

The original query was:

In /sys/dev of 4.1BSD, there is a driver for the pseudo-device pty.
The only description of it is in the source - no manual page, etc.:

 * A pseudo-teletype is a special device which is not unlike a pipe.
 * It is used to communicate between two processes.  However, it allows
 * one to simulate a teletype, including mode setting, interrupt, and
 * multiple end of files (all not possible on a pipe).	There are
 * really two drivers here.  One is the device which looks like a TTY
 * and can be thought of as the slave device, and hence its routines
 * are prefixed with 'pts' (PTY Slave).	 The other driver can be
 * thought of as the controlling device, and its routines are prefixed
 * by 'ptc' (PTY Controller).  To type on the simulated keyboard of the
 * PTY, one does a 'write' to the controlling device.  To get the
 * simulated printout from the PTY, one does a 'read' on the controlling
 * device.  Normally, the controlling device is called 'ptyx' and the
 * slave device is called 'ttyx' (to make programs like 'who' happy).

Has anyone ever used it? What type of applications could it be used for?
And (most importantly) does it work?

====================

That "pty.c" is used INTENSIVELY on the Purdue/ECN networking; I believe
that is where Berkeley got it. (It's used for network logins and I believe
that 4.2 networking uses it...)

The idea is that you have a networking demon that talks to a network socket
as so:

	<user>	<cu-like process> <local socket> <remote socket> 
		<cu-like process> </dev/pty00> <remote pty driver>
		</dev/ttyp0> <login shell>

The "cu-like process" on either machine just copies "from here to there",
possibly sending signal information (he typed a DEL, send a DEL to the
process group on the other end) and the like to the other side, like a 
pipe.
======================
	I hate not to be of help.  I am looking at putting that
driver on a Sys III variant and I would also like to know
applications.  I know of only two, and not in detail yet.
4.2 BSD uses pty's to implement remote login over LAN, and
I have a Chaosnet application that used them for tn(1) (TELNET
access).
======================

The pseudo-teletype ("pty") is not a Berkeley-ism, nor even a UNIX-ism.
It's pretty widely industry standard on timesharing operating systems.
(My first exposure to them was back in 1970, on DEC PDP-10's, but they were
old even then. They pronounced the acronym "pity", to rhyme with "tty".)
((My apologies if I leave out earlier systems, maybe MULTICS, of which
I have no direct experience.))

A pseudo-teletype is a specialized form of a "virtual device", that is,
a "device" which is really manifested by a program ("Pay no attention to
the [process] behind the curtain!"). As said in the excerpt you quoted,
a pty normally has two ends, a "master" end and a "slave" end. Under UNIX,
the "master" end is normally named "/dev/ptyNN" while the slave end is named
"/dev/ttyNN". (On some systems the slaves are "/dev/ttypN".)

The point is that since (if the implementation of "pty.c" is correct, and
it should be) there is no way that a process can (easily) tell that the
"slave" end is not a real, live, physical piece of hardware, and a process
controlling the "master" end can EXACTLY emulate a physical human sitting
at a terminal typing and reading. In fact, this emulation is generally so
exact that the slave ends are simply listed in /etc/ttys with all the other
terminals, and get a "getty/login/shell" from "init" just like the rest,
with a line in /etc/ttys like "1mtty03" or "1mttyp3" (login port, 9600
baud, /dev/tty03 or /dev/ttyp3). [Here on, assume slaves named ttyNN.]

So (messy details of flag handling and synchronization omitted), some
process "X" opens the master side of a pty, say /dev/pty43. Since the
beginning of time, some copy of the program "getty" was trying to open
/dev/tty43 (since "init" forks a copy of "getty" for each line in /etc/ttys),
and that open now completes, since "somebody" (process X) just "dialed in".
"Getty" writes a banner on tty43, which X reads by reading pty43. X then
"types" its login name by writing on pty43 which "getty" reads on tty43,
and so on through the login process, running the shell, and then the user
programs, etc.  "Terminal input" is "typed" by X by writing on pty43,
"display output" is "read" by X by reading pty43, and all the user programs
are dumb and happy thinking there's a human on the other end of tty43.

So why would one want to do this? Basically, in historical order of
evolution, four reasons:

1. Batch (Background) Processing of Commands. On systems (not UNIX) which
   are NOT process oriented, and which therefore did NOT have the concept
   of a "fork", there was no easy way to have background (or "batch") jobs.
   The pty was a clever, elegant solution that left the basic "timesharing"
   nature of the system unchanged, while providing essentially what we now
   call shell scripts. Example: On a PDP-10, you said ".submit foo.ctl" and
   "foo.ctl" was fed to an operator program named BATCON (run from one of the
   operator's consoles) that opened a pty and "typed" your monitor commands
   in (including logging you in and out), and saved up the output in a file
   called "foo.log" for your later amusement and amazement (since Murphy's Law
   was true even back then ;-}).

Even with all that convenience (;-}), systems administrators got tired of
having a terminal in the machine room for EACH copy of EACH operator service
program (no "fork", remember, so one tty for each line printer spooler and
for each batch stream and for each card reader...), so...

2. Job Control - Multiple jobs active from one terminal. A new operator service
   program (named OPSER on PDP-10's, OPRCON on TOPS-20) was invented which
   incestuously used ptys to control all of the other operator service jobs!
   You could type a line into a given job by prefixing the line with the
   subjob number or mnemonic (set up by the operator), such as, to change
   forms on printer 3 you said "L3-forms payroll2" (or something). Since it
   was perfectly general, the operator could also do small utility tasks
   (like accounting, etc.) through the same terminal (except for the hassle
   of those other jobs sometimes yammering at you -- but there were commands
   to handle that). For convenience, if you typed a line that didn't start
   with "XX-" it assumed you meant the last one you explicitly talked to,
   so an extended conversation was fairly confortable.

   Eventually, ordinary users started using OPSER when they needed to multiplex
   several activities, or whenever they needed its script and logging features.
   (Like the "script" program.)

In modern terminology, what you had there was a line-oriented "window" manager,
similar to the "Maryland Windows" package, but for Teletype Model 33's.
Or, somewhat like Berkeley 4.1bsd job-control under the C-shell ("csh"), but
with processes NOT suspended just because you weren't typing at them (they
could still type out progress reports, if you didn't disable them).

	[Personal aside to any wizards who have made it this far:
	 having used both OPSER and 4.1 csh with job-control, I'll
	 take pty-based "job control" hands down, PROVIDED it's
	 always there and is the default. I understand the problem
	 in S5 is you have to decide ahead of time whether you're going
	 to need it (?).]

Home stretch...

3. Network Virtual Terminals. With the advent of networks of many strange
   and wonderful kinds, seldom being integrated into the operating system
   completely on first release, the ancient pty was again drawn into service
   [is this getting a little thick? ;-}] as a quick, simple, effective way
   of implementing a basic network service -- remote logins. The operating
   systems (including UNIX) rarely treated network connections as full
   citizens, and besides, there were many versions -- which one(s) to support?
   The answer (even today, often) is to have a program running on the target
   system (the one you're trying to login on) called a "network virtual
   terminal protocol server" (typically named "vtpsrv") which sits around
   waiting for people out in the network to connect to it (via whatever net
   hardware and software one might have). When someone does so, the "vtpsrv"
   handles all the network stuff, eventually passing characters typed by a
   user (or a program) out in the network to the master side of a pty as
   if the user were local to the target machine, and "displaying" output
   (read from the pty) by sending it back out into the network. Because
   of the local (target) "vtpsrv", the target operating system doesn't
   have to worry about "network terminals" -- it doesn't have any! (That's
   the "virtual" part.) Smarter versions of "vtpsrv" (or of the corresponding
   program that the real user runs out there in the network, usually called
   "vtp") can sometimes also handle some transformations from one terminal
   type to another (a different meaning of "virtual terminal").

   Naturally, there should be some standards around for how the various
   pieces talk to each other. There are, but of course they are network
   dependent. Within the ARPAnet (or other nets using IP/TCP protocols),
   the protocol between a "vtp" and a "vtpsrv" is called "Telnet" (NOT to
   be confused with the company Telenet!). Within the CCITT world (the "X.nn"
   and "V.nn" international standards), the protocol between the user and
   "vtp" (yes, users have to obey protocols, too!) is called X.28, the protocol
   between "vtp" and "vtpsrv" is called X.29, and the overall scheme is called
   X.3 (and sits on top of X.25 -- aren't you glad you asked?). Telenet (the
   company) uses protocols VERY close to the CCITT standards.

Of course, as efficiency becomes an issue, it is tempting to put all of
"vtpsrv" in the kernel of the operating system, and this is often cited
as a mark of a "integrated" network offering. Nevertheless, the flexibility
of the pty approach often wins over raw efficiency. (Various players do it
various ways.)

Finally (and this IS short),

4. Virtual Devices. Although the pty is a specialized virtual device, having
   (usually) just the set of "ioctl" calls needed to emulate a tty, sometimes
   people will use them to emulate hardware that hasn't been built yet (or
   for which the driver hasn't been written yet), in order to decouple the
   writing/building of the "system" side (driver/hardware) from the "user"
   side (applications programs). With a little tinkering (mostly just more
   general handling of ioctl's), ptys can be made to emulate any UNIX
   "character device". This hack has even been used occasionaly to execute
   programs that would only run on specific hardware or kernels, when
   equivalent (but unfortunately, different) systems were available.

SUMMARY

These days, #3 (networks) is the most common use of ptys, but the increasing
emphasis on "windows" is reviving #2 (multiplexing).

p.s. I have completely ignored the issue of the so-called "mpx" device,
     found (partially) in UNIX Edition 7 (v.7) (and other UNIXs, as well as
     several non-UNIX systems), which could be used like a pty with one master
     side and many slave sides. Requicat In Pacem.  Who knows, it may become
     "popular" again some day, eh?
==================

Two existing applications that I know are 1) as a driver for telnet
connection; 2) as a driver for Mitrenet telnet service.  BSD 4.2 has 
the telnet command that allow you to access remote computer through
telnet connection.  The telnet and pty were written by BBN for
terminal-to-process, process-to-process, terminal-to-host communication.
Try typing "telnet" to see if your UNIX has it.

==================
Yes, the 4.1 BSD ptys work most of the time.  But there is a serious
bug, for which I posted a report to the network roughly a year ago.
If you close the controller end while a process is writing on the slave,
the writing process may be re-entered at location 0 in kernel mode.
It has been fixed in 4.2.
==================
Yes, ptys work. Where I used to work, we had Xerox Lisp machines on an
ethernet, and the net software allowed one to login from the net. It used
ptys to do this. I don't know whether the version of pty.c that you have is
the same as what we had, but I only know of one version, so it will probably
work for you.

It has been a while, and I didn't really understand what I was doing when I
installed them, so I can't give you much info on how to do it. I do remember
that the ptys are called pseudo devices (in the file whose name is given as
arg to /etc/config), and that there are two major devices (control/slave)
with all the individual ptys as minor devices under them. You make as many
/dev/pty0, /dev/pty1, etc. as you make /dev/ttyp0, /dev/ttyp1, etc. The minor
device numbers match up each controlling pty? with the corresponding ttyp?.
(I think we actually called them /dev/ttyp[a-z] and /dev/pty[a-z]).

If you want, you can specify the /dev/ttyp? in /etc/ttys, so /etc/init will
run /etc/getty on them and you can login by opening/reading/writing on
/dev/pty?.

================
The supplied pty driver works.  It has a bug:  as distributed it
only runs OTTYDISC (i.e. "stty new" works but has no effect).  There
is a very simple fix (find the place(s?) where it calls the old
tty routines directly and change the code to call the appropriate
routine based on the line switch table and tp->t_ldisc).

If you make nodes for these ptys and put them in your init table,
you can log in by opening the control name.  This is how the BBN
"telnet" program for 4.1 works.

=====================
The pty driver is a hack to allow a user-mode process to present a TTY
interface to another program.  It was used by the old ARPAnet NCP Telnet
protocol, and most recently by the 4.1BSD/BBN TCP/IP Telnet protocols.

The model is as follows:

___________      _________________
| Network | <--> | Telnet Server | <---> /dev/ptcN  :c:l:i:s:t:s /dev/ptyN <--> SH
-----------      -----------------

Essentially, the telnet server reads characters from the network, and
writes them to the PTY controller (here, /dev/ptcN.) This action is
analogous to a DH or DZ interrupt routine, in that it puts characters on
the terminal's input queue (here, the PTY "slave", /dev/ptyN).  The user
program (here, SH) will then receive them, by reading from its standard
input.  In the same way, when the user program writes to its standard
output (also open on /dev/ptyN), the telnet server will be able to read
those characters from the corresponding pty controller device, and then
write them onto the network connection.  This is roughly analogous to the
"output interrupt" processing of a DH or DZ.  The key is that this "server" 
can do anything it likes to the characters it reads or writes on the
controller.  Here, it is essentially a pass-thru (excepting the minor Telnet
negotiations and protocol) serving merely to hook up a network connection
to the UNIX model of a TTY.  The PTY "slave" looks exactly like any TTY to
the process which has it open, and indeed, one would usually enter such
pseudo-TTY's into the /etc/ttys file.  Virtual carrier is set when the
corresponding controller device is opened.

You could, instead, use the PTY's for anything you like, not necessarily
a network.  I once wrote a special filter for someone who had a DG "Dasher"
terminal which went wild when it was sent ^H characters.  We set her shell
to read from a pseudo-TTY, and the process reading and writing on the
controller mapped the characters appropriately.

===========
In addition to replies to my question, the following question was once
posted to the net:

Could someone who is familiar with the pty driver answer a question for me?

I tried installing some ptys on my 4.1BSD system, and then used them by putting
a getty on ttya and doing 'cu ttya'.  It works fine for a minute or two,
allowing me to get logged onto the pty and type a few commands.  Then, suddenly
cu thinks it lost the line.  Turns out one of the reads on the controller end
returned a -1, with errno set to ENXIO.  ("No such device or address.")  I
couldn't see anything in the driver that could generate this.  I changed cu
to ignore the error condition and it seemed to work OK.

What would cause a pty controller device to return a read error?  What is
the standard technique for programming around this - just ignore the error?
This seems dangerous - perhaps only ENXIO's should be ignored.  Anyone know
what possible errors there might be?