Standards Update, IEEE 1003.3: Test Methods

[Jeffrey S. Haemer]

Submitted-by: jsh at usenix.org (Jeffrey S. Haemer)

           An Update on UNIX1-Related Standards Activities

                           October 1, 1990

                 USENIX Standards Watchdog Committee

                   Jeffrey S. Haemer, Report Editor

IEEE 1003.3: Test Methods

Doris Lebovits <lebovits at attunix.att.com> reports on the July 16-20
meeting in Danvers, MA:

Overview

Dot three's job is to do test methods for all of the other 1003
standards.  The group's work, whose first parts are now in ballot,
specifies the requirements for OS conformance testing for our industry
and for NIST.  This makes our balloting group, our technical
reviewers, and our schedules worth watching.  Pay attention, also, to
what comes out of the Steering Committee on Conformance Testing
(SCCT).  Their projects and decisions will be interesting and
important.

This was the working group's seventeenth meeting.  As usual, we
reviewed the ballot status of P1003.1 test methods, worked on P1003.2
test methods and reviewed steering committee activities.  Technical
reviews were done on parts I and II and the group developed assertions
for part III.  Participants from the usual companies attended (AT&T,
NIST, OSF, Mindcraft, IBM, DEC, HP, Data General, Cray Research,
Unisys, Perennial, and Unisoft, Ltd.), as did an assortment of P1003.2
members (see below).

Document structure

Currently, our evolving document has three parts: Part I is generic
test methods, Part II is test methods for measuring P1003.1
conformance, including test assertions, and Part III contains test
methods and assertions for measuring P1003.2 conformance.

After the ballot, each part will become a separate standard.  Part I
will be published as IEEE P1003.3, Part II as IEEE P1003.3.1, and Part
III as IEEE P1003.3.2.

__________

 1. UNIXTM is a Registered Trademark of UNIX System Laboratories in
    the United States and other countries.

October 1, 1990 Standards Update             IEEE 1003.3: Test Methods


				- 2 -

Ballot status

Draft 11 of the current ballot, which was recirculated to the
(approximately) ninety-member balloting group late in February, closed
balloting March 23.  Of the respondents, 19 disapproved with
substantive negative comments.  This met the two-thirds response
requirement, but falls short of the needed two-thirds approval.

A recirculation ballot for P1003.3 Draft 12, which is the revision of
Part I of Draft 11, began August 28 and is expected to close September
28, 1990.  The recirculation of P1003.3.1 Draft 12 (Part II) will be
conducted at a later date.

On the first and last days, the technical reviewers worked on ballot
objections to Part I and Part II.  All Part I objections and most Part
II objections were resolved.  The definition of an untested assertion
was reviewed and a permanent rationale will be included in Part I.

P1003.2 verification

This was our fifth meeting working on the verification standard for
the P1003.2 standard.  The assertion writing and review were done
jointly with the P1003.2 working group.

The whole P1003.3 and P1003.2 working groups worked jointly on
defining test assertions based on P1003.2 Draft 10.  They worked in
three small breakout groups.  The joint group (P1003.2 plus P1003.3)
also met in plenary session several times to discuss progress and
small-group issues.  Progress was slow in the beginning, since most of
the P1003.2 working group were not familiar with test assertions.  but
by the end of the week we had discussed and resolved several issues.
Some examples:

   - Do we need to state assertions in P1003.3.2 explicitly that
     duplicate P1003.3.1? (Yes.)

   - Must we test locale variables for every locale-sensitive
     interface?  (They should be tested when their behavior is clearly
     stated for a utility.)

   - Should assertions for multiple operands be consistent? (Yes.)

Lowell Johnson (Unisys) is Secretary of the P1003.2 Test Methods
activities, and Andrew Twigger (Unisoft Ltd) is Technical Editor.  Ray
Wilkes, the former Chair, has changed jobs and is no longer able to
attend regularly, so Roger Martin is actively looking for a
replacement.

October 1, 1990 Standards Update             IEEE 1003.3: Test Methods


				- 3 -

Steering Committee on Conformance Testing (SCCT)

The SCCT is supposed to alleviate the increasing dot-three work load
that all the other proliferating groups are creating.  Their job is
coordinating the activities of all test-methods groups, monitoring
their conformance to test methods, and writing Project Authorization
Requests (PARs).  Currently, its members are Roger Martin (NIST,
Steering Committee Chair), Anita Mundkur (HP), Andrew Twigger (Unisoft
Ltd), Bruce Weiner (Mindcraft), Lowell Johnson (Unisys) and the newest
member, John Williams (GM).  That there is a new member in the
steering committee is very important, especially because John is from
GM, the largest user voice other than the U.S. government.

The steering committee did not have anything for the working group to
review.  It is still documenting procedures, and Roger is still
clarifying which standards the working group will address.

October 1, 1990 Standards Update             IEEE 1003.3: Test Methods

Volume-Number: Volume 21, Number 162