Standards Update, IEEE 1003.2: Shell and tools

[Jeffrey S. Haemer]

From: Jeffrey S. Haemer <jsh at usenix.org>


            An Update on UNIX* and C Standards Activities

                            December 1989

                 USENIX Standards Watchdog Committee

                   Jeffrey S. Haemer, Report Editor

IEEE 1003.2: Shell and tools Update

Randall Howard <rand at mks.com> reports on the October 16-20, 1989
meeting in Brussels, Belgium:

Background on POSIX.2

The POSIX.2 standard deals with the shell programming language and
utilities.  Currently, it is divided into two pieces:

   + POSIX.2, the base standard, deals with the basic shell
     programming language and a set of utilities required for
     application portability.  Application portability essentially
     means portability of shell scripts and thus excludes most
     features that might be considered interactive.  In an analogy to
     the ANSI C standard, the POSIX.2 shell command language is the
     counterpart of the C programming language, while the utilities
     play, roughly, the role of the C library.  POSIX.2 also
     standardizes command-line and function interfaces related to
     certain POSIX.2 utilities (e.g., popen, regular expressions,
     etc.) [Editor's note - This document is also known as "Dot 2
     Classic".]

   + POSIX.2a, the User Portability Extension or UPE, is a supplement
     to the base POSIX.2 standard; it will eventually be an optional
     chapter of a future draft of the base document.  The UPE
     standardizes commands, such as screen editors, that might not
     appear in shell scripts but are important enough that users must
     learn them on any real system.  It is essentially an interactive
     standard that attempts to reduce retraining costs incurred by
     system-to-system variation.

     Some utilities, have interactive as well as non-interactive
     features In such cases, the UPE defines extensions from the base
     POSIX.2 command.  An example is the shell, for which the UPE
     defines job control, history, and aliases.  Features used both
     interactively and in scripts tend to be defined in the base

__________

  * UNIX is a registered trademark of AT&T in the U.S. and other
    countries.

December 1989 Standards Update            IEEE 1003.2: Shell and tools


                                - 2 -

     standard.

In my opinion, the biggest current problem with the UPE is that it
lacks a coherent view: it's becoming a repository for features that
didn't make it into the base standard.  For example, compress is in
the current UPE draft.  It's hard to rationalize classifying file
formats as an "interactive" or "user portability" issue, yet the one
used by compress is specified in the UPE.  It certainly doesn't fit in
with a view of the UPE as a standard that merely adds utility syntax
information (e.g., information that would allow users to type the same
command line to compress a file on any system).  This highlights the
schizophrenic nature of the UPE: it addresses a range of different
needs that, taken together, do not appear to define a whole.  Dot 2
Classic, to my taste, appears to have far more unified scope and
execution.

A second, related, problem with the UPE is that there appears to be
less enthusiasm for it than for the base standard.  A number of
people, including me, understand the need for it, but it doesn't
appear to have the strategic importance of the base.  [Editor's note -
The UPE is, frankly, controversial.  Like 1201, the committee
undertook the UPE out of a fear that if they didn't, NIST would do the
job without them.  Supporters note that although its utilities are
probably not necessary for portability of most software, it would be
unpleasant for programmers to do the porting work without them.
Detractors counter that POSIX was never intended to cover software
development and that the group is exceeding not only its charter, but
that of the entire 1003 committee.]

Status of POSIX.2 Balloting

POSIX.2 is in its second round of balloting.  The first ballot, on
Draft 8, produced many objections that are only partially resolved by
Draft 9.  Although there were only fifty-four pages of unresolved
objections remaining after Draft 9 was produced, the current balloting
round is not restricted to existing objections, and there will almost
certainly be many new ones.  Remaining objections range from the
perennial war between David Korn and the UNIX Support Group over what
features should be required in the POSIX shell, through the resolution
of the incompatible versions (Berkeley and USG) of echo, to the
treatment of octal and symbolic modes in umask.

A digression to illustrate the kind of issues being addressed:

     In March of 1989, a study group from 1003.2 met at AT&T to
     resolve major objections to the shell specified in Draft 8 by the
     two warring parties.  This was a good place to hold the meeting,
     since both parties are from AT&T: one led by David Korn of Bell
     Labs, the author of the popular Korn Shell (KSH) the other, a
     group led by Rob Pike of Bell Labs Research and the UNIX Support
     Organization, advocating more traditional shells, like the System

December 1989 Standards Update            IEEE 1003.2: Shell and tools


                                - 3 -

     V Bourne Shell and the Version 9 Research shell.  Korn's group
     contends that the shell should be augmented to make it possible
     to efficiently implement large scripts totally within the shell
     language.  For example, while the more traditional camp views
     shell functions as little more than command-level macros and uses
     multiple scripts to modularize large shell applications, the Korn
     shell views functions as a tool for modularizing applications,
     and provides scoping rules to encourage this practice.

     The two philosophies engender different opinions on issues such
     as the scoping of traps within functions and the use of local
     variables.  Other contentious issues were the reservation of the
     brace ({ }) characters as operators (rather than as the more
     tricky "reserved words"), the promotion of tilde expansion to a
     runtime expansion (like parameter expansion), and the issue of
     escape sequences within echo/print/printf.

     The meeting produced a false truce.  I attended, and believe that
     both parties had different views of the agreement that came out
     of the meeting.  As a result, Draft 9 produced balloting
     objections from both parties and the dispute continues unabated.
     Shades of POSIX.1 Tar Wars...

I suspect the next draft (Draft 10) will fail to achieve the consensus
required for a full-use standard.

This is a good thing.  Useful features are still finding their way
into the document.  (Draft 9 introduces hexdump, locale, localedef,
and more.) Also, the sheer size (almost 800 pages) of Draft 9 has
prevented many balloters from thoroughly reviewing the entire
document, Still, there is a stable core of utilities that is unlikely
to change much more than editorially; I predict the standard will
become final around Draft 12.

A mock ballot on Draft 4 of the UPE will probably start after the New
Orleans meeting in January, and the resulting Draft 5 will probably go
to a real ballot somewhere in summer to early fall of 1990.  Although
many sections remain unwritten or unreviewed, the UPE is a much
smaller standard than POSIX.2 and should achieve consensus more
quickly.

Status of the Brussels Meeting

The Brussels meeting focused on the UPE, with only a summary report on
the status of balloting for the base standard.  For most of the
meeting, small groups reviewed and composed UPE utility descriptions.
The changes generated at the meeting will appear in Draft 3.

The groups reviewed many utilities.  The chapter on modifications to
the shell language (for interactive features) is now filled in, and
such utilities as lint89 (the recently renamed version of lint), more,

December 1989 Standards Update            IEEE 1003.2: Shell and tools


                                - 4 -

etc.  are approaching completion.  Still, much work remains.

[Editor's complaint - We think renaming common commands like lint
("lint89") and cc ("c89") is both cruel and unusual.  We are not eager
to re-write every makefile and shell script that refers to cc or lint,
nor to re-train our fingers to find new keys each time the C compiler
changes.  The name seems to have been coined by either a hunt-and-peck
typist, or someone who has longer and more accurate fingers than we
do.  (Was it, perhaps, the work of Stu Feldman, author of f77?)
Moreover, replacing commands with newer versions is commonplace and
traditional in UNIX.  Examples like "make", "troff", and "awk" spring
to mind.  If an older version is kept on for die-hards, it's renamed
(e.g., otroff, oawk).

One Dot-Two member rebuffed our objections with the reply, "But, you
see, this isn't UNIX: it's POSIX." ]

Because the meeting was in Europe, attendance at the working group
meetings was lower than normal (20-25 rather than the normal 35-40 in
POSIX.2.  Nevertheless, the choice of location served a purpose.  The
meeting was held in Brussels to garner international support and
participation, particularly from the European Economic Community.
There were many EEC representatives at the background sessions on
POSIX and two or three European working group members in the POSIX.2
meetings who wouldn't normally have attended.  Though it remains to be
seen what will come out of having met in Brussels, I am convinced that
the extra effort will prove to have been justified.

December 1989 Standards Update            IEEE 1003.2: Shell and tools


Volume-Number: Volume 18, Number 4