mod.std.unix P1003 job control proposal (now vhangup, auto-nice)

[Moderator, John Quarterman]

From: seismo!munnari!yabbie.rmit.oz!rcodi at sally.utexas.edu (Ian Donaldson)
Date: Tue, 4 Nov 86 09:05:35ligt 

> From: pyramid!nsc!hplabs!hpda!hpisoa1!davel (Dave Lennert)
> Date: Wed, 29 Oct 86 10:07:47 -0800

I spoke about HUP's sent when an _exit is done, not realising at the time
that it is only done by a sesion-group leader, and only to
the same process-group, obviously (it is now) excluding those
processes running in background.

> This key feature is preserved in the POSIX specification.  Perhaps I
> don't understand your statement.

I withdraw the statement.

In the same article, I wrote: 

> > vhangup() will provide clean terminals on a bsd system,
> > and we have improved vhangup further to not just turn off READ/WRITE bits,
> > but to actually redirect the file references to /dev/null (which has
> > the advantage of also dropping DTR reliably). 
> 
> vhangup() is indeed desirable.  It is not required in POSIX because it is 
> not supported on System V and, indeed, breaks System V compatiblity.

I forgot to add the other advantage of my vhangup(), in that
any read's by processes from the terminal return EOF rather than
a read-error, and any writes go to the bit-bucket, rather than
producing a write-error.  This is very desirable in stuations where
you have started something in foreground (eg: make, without redirecting I/O) and
you want it to -complete- in background, rather than crash because
it cannot write diagnostics or output...
Of course, you cannot -see- the output, but that is another topic
(along the lines of login-suspension - see ravings in net.unix-wizards
about 6 months ago).

Some programs of course would need to be trained to not repeatedly
read beyond EOF assuming somebody is hitting ^D, otherwise they
will do-so forever.  Berkeley 4.2bsd Csh and Mail already have been
trained, and exit if a certain large number of consecutive EOF's are read.
There are probably others.
We have not felt need to modify any more programs, though there is the stray
program that just sits there all day, once a fortnight perhaps.
Readnews seems to be an often offendor (2.10.3).

I also wrote:

> > Infinite-loop processes don't cause problems with system-response because
> > they are automatically niced (something that is long-needed in UNIX 
> > systems).
> 
> Note that this is NOT desirable on, e.g., realtime systems where that
> long running process may be critical.  I'm also getting tired of having
> my rwhod reniced automatically.
> 
>     Dave Lennert                ucbvax!hpda!davel               [UUCP]

Granted, it probably -isn't- desirable at some sites, or perhaps even
some particular users at certain sites, but it is necessary for
any site that has lots of cpu-intensive tasks mixed in with lots
of heavily interactive (eg: vi) sessions.

The auto-nicing that I have seen in 4.2 (not sure who added it) whereby
a process gets niced by 1 every 40 cpu seconds until the nice reaches +15
is NOT the sort of thing I am talking about.  The obvious problem with
this algorithm is that it makes no distinctions on what it nices -
it nices compiles (ok, good), but it also nices vi (bad), and eventually
nices the shell (worse, as everything you start inherits the nice
from the shell).  It also makes no distinction on -who- it nices;
except that root is never niced (neither are other users, provided
they are running at elevated priority).  This means that all non-root owned
daemons get niced also.

I experimentally implemented a smarter, but still very simple auto-nicing
algorithm that works well here nearly all of the time:  it is an
extension of the one above, except that interactive processes are
identified very simply by the fact that they do tty input (sounds logical).

A process is only niced if it consumes N cpu seconds since last doing
some tty input (we have N set small, to 5 cpu seconds). 
This means that an interactive shell is rarely ever
niced, and vi always has a good response time regardless of what
else is going on in the system (unless you're doing complicated
length string searches or the like).  Another "quick hack" allowed
all uid's less-than a certain number (say 2) to be immue to
auto-nicing - this got around the daemon problem, altough very
unelegantly.

The kernel mods were trivial, amounting to less than a screenful
of code all-up.  In a nutshell it was one mod to clock.c 
(kern_clock.c for 4bsd), to increment the nice if the conditions
warrant it; and another mod to ttyread() to reset the nice
and the accumulated cpu time since the last ttyread().

I have watched people breath discontent over the net with vi's response time, 
and all the hack's that they've tried (including a suid root pre-elevate-nice 
command).  Vi is only mentioned because it is common to almost all
systems.  The problem is much deeper than just vi.

Any system that treats non-interactive jobs the same as interactive jobs
is sure to have a rotten response-time.  Interactive jobs are not
jobs that are just "associated" with a terminal either.  People
expecting to do half-hour compiles from the terminal soon learn
that there is no advantage in doing-so, and it is better to come back
later and free up the terminal for somebody else to use (productively).

> Note that this is NOT desirable on, e.g., realtime systems where that
> long running process may be critical.  

Perhaps, but then my version of auto-nice may solve most of such
problems anyway.  If the program on the real-time system communicates
with other than tty's in an "interactive" fashion, then this could be 
incorporated into the algorithm quite easily.  Lots of things could be 
incorporated into the algorithm - such as a per-user nice-rate and
nice-limit.

People will never learn - just telling them to "nice" their jobs
rarely works.  They either forget to, or just don't do it on principle.

SVR2 has no way of renicing a job other than restarting it
(counter-productive), and telling somebody to change their nice on the
humungous compile they just started after they've left and gone home is
also often very difficult (read impossible), leaving a very slow
system.

Vi and similar programs on our systems nearly always have a
good response time - the times that they don't would most probably
be attributed to excess paging and swapping activity and filesystem I/O.

> I'm also getting tired of having
> my rwhod reniced automatically.

Maybe it wouldn't be with the algoritm I mentioned (I've never used
rwhod so I can't answer that).

At least auto-nicing of some variety I have mentioned should
be provided as a standard -option-.  Those who don't like it
can turn it off, but I'll bet your boots that they will be
in a minority once they've tried it.  Not having the option
at all is not very nice (excuse the pun :-).

Ian Donaldson

Volume-Number: Volume 8, Number 37