UNIX-WIZARDS Digest V5#096
Mike Muuss
Unix-Wizards-Request at arpa.brl
Wed Jul 13 17:45:24 AEST 1988
UNIX-WIZARDS Digest Wed, 13 Jul 1988 V5#096
Today's Topics:
Re: Getting the pathname from a FILE*.
setsockopt -- SO_LINGER parameter?
Re: Interactive Background Processes
wanted: gdb for system V.2 (68010)
Re: Getting the pathname from a FILE*.
Re: I/O Processors & Smart Screens
Re: Who dat?
Re: setsockopt -- SO_LINGER parameter?
BSD 4.3 F77 fortran query
Re: How do I get RCS?
Re: BSD 4.3 F77 fortran query
setgid and process group list
Re: Realtime OS's
Salvaging info on a streamer tape
Re: BSD:sockets::SVID:(what?)
undump
Is write(2) "atomic" ?
UUCP
File space allocation/deallocation under Unix (BSD4.x)
Re: Is write(2) "atomic" ?
Re: File space allocation/deallocation under Unix (BSD4.x)
Re: IBM speed
UUCP
Re: Is write(2) "atomic" ?
Anyone running 4.3+NFS from U of Wisc ?
Re: File space allocation/deallocation under Unix (BSD4.x)
Re: Is write(2) "atomic" ?
where to do line editing?
Re: Is write(2) "atomic" ?
Apologies
Re: Getting the pathname from a FILE*.
Input Line Editing
-----------------------------------------------------------------
From: Juergen Wagner <gandalf at csli.stanford.edu>
Subject: Re: Getting the pathname from a FILE*.
Date: 11 Jul 88 05:25:30 GMT
To: unix-wizards at SEM.BRL.MIL
In article <1645 at osiris.UUCP> guest at osiris.UUCP (Guest account) writes:
>...
>*PLEASE* - if you are going to do things like using internal elements of
>"standard" libraries, try to access them in a more or less "standard"
>way. Most stdio libraries I've seen have a macro (but could be func, I
>guess) called fileno(myfd) - which returns just the information you are
>referring to. This way, your code won't break if someone does something
>cute or clever to the stdio library and include files.
I am aware of the fact that there is fileno(f). The point is: there
is a way to find out the file descriptor associated with FILE *f.
>Does this work if the file system is NFSsed, or something like that ?
If you do it properly: yes. Otherwise: no! The problem with NFS file systems
is that you have to walk through something other than the local file system.
I have a small program which determines the file system a file resides on,
and in case of NFS just returnes the remote host's name and the remote file
system's name. This allows you to walk through the directories as before.
At the time I posted my article, I didn't think of files being deleted while
they are still open somewhere else... This shouldn't cause any problems because
your function determining the file name could then just say so. Sockets, pipes,
ptys, and other stuff like raw devices can be handled with fstat. The inode
type will give you the information needed.
As for files having multiple links, I can only ask the question "what do you
want the routine to do?" Do you want a list of all names the file can be
accessed under, or are you merely looking for some way to associate this
FILE * structure with *SOME* file name on your UNIX system? Personally, I
would be happy with the latter solution. If you get your hands on one of the
file's names, you've got the file in your grip!
>It almost seems to me that this is a case which stdio is not designed to
>handle. So - I'd say add another level of indirection. Write a routine
>that acts like fopen() - and one that acts like fclose(), etc. Have it
>store the name someplace, using only the "standard" interface to stdio,
>and then write another routine, or macro, that allows you to get the
>information back. I can't imagine that would be hard, and I expect it
>will work a LOT faster than reading your file system.
That's not always possible, as I've explained above. You can only find out
the names of files *YOU* have opened yourself. You will never be able to
find out what stdin/stdout/stderr are connected to...
>Then again, maybe I don't understand what you're trying to do.
I guess, understanding a tool is not a necessary precondition for being able
to use this tool. I can fairly well work with UNIX but don't claim to have
understood all the bits of it! The issue is to provide some function which
performs its task in a way opaque to the user. One reason to do so is that
what this function really does is operating-system dependent. And there we
are again at the same point where you flamed at me because I preferred to
mention _file instead of fileno(f). Opaqueness is a very important means of
keeping things understandable. If somebody knew that a certain function would
work more efficient under some circumstances than under other, he/she would
use that knowledge in his/her programs. However, you know as well as I do that
this may turn out to be much less efficient on other systems.
# include <disclaimers/strawberry.h>
--
Juergen "Gandalf" Wagner, gandalf at csli.stanford.edu
Center for the Study of Language and Information (CSLI), Stanford CA
-----------------------------
From: Steven Grimm <koreth at ssyx.ucsc.edu>
Subject: setsockopt -- SO_LINGER parameter?
Date: 8 Jul 88 16:25:07 GMT
Sender: usenet at saturn.ucsc.edu
To: unix-wizards at SEM.BRL.MIL
Does anyone know what kind of value setsockopt() is expecting
as a parameter to the SO_LINGER flag? I can't find it documented
anywhere. I'm guessing it's a struct timeval, since that seems to
be used a lot in system calls. Does anyone know for sure?
---
These are my opinions, and in no way reflect those of UCSC, which are wrong.
Steven Grimm Moderator, comp.{sources,binaries}.atari.st
koreth at ssyx.ucsc.edu ...!ucbvax!ucscc!ssyx!koreth
-----------------------------
From: "John F. Haugh II" <jfh at rpp386.uucp>
Subject: Re: Interactive Background Processes
Date: 9 Jul 88 14:46:15 GMT
Posted: Sat Jul 9 10:46:15 1988
To: unix-wizards at SEM.BRL.MIL
In an earlier posting I was discussion background processes in 5.0 and
mentioned how not all implementations were capable of running multiple
background jobs. Someone from SCO sent me a letter and here is the
text of it:
+----
|> unfortunately, not all of the modern unix implementations have sxt's and
|> the shell layer manager. for example, the plexus doesn't have any of
|> those things. the wyse, running xenix, does, but only has one control
|> sxt so only one instance of shl can be running.
|>
|>- john.
|>--
|
| If you want to run more than one instance of shell layers,
| run configure(C) and raise NSXT from its default of 1.
| No inherent limitation here.
| --amy
+----
--
John F. Haugh II +--------- Cute Chocolate Quote ---------
HASA, "S" Division | "USENET should not be confused with
UUCP: killer!rpp386!jfh | something that matters, like CHOCOLATE"
DOMAIN: jfh at rpp386.uucp | -- with my apologizes
-----------------------------
From: Paul Lew <lew at gsg.uucp>
Subject: wanted: gdb for system V.2 (68010)
Date: 9 Jul 88 18:05:49 GMT
Posted: Sat Jul 9 14:05:49 1988
To: unix-wizards at SEM.BRL.MIL
Environment:
OS: ATT system V release 2.4
(some modication to support bsd TCP)
CPU: 68010
GDB source version: 2.5
wanted ---->>> m-sysV.h (or some other name)
I noticed there is a m68k-opcode.h in gdb distribution which should be
usable, but there is no m-<machine>.h file for system V. Could someone
mail me a copy if you have it? (RMS told me there exists a port of gdb
to system V, but he hasn't see the changes yet)
Any help or pointer will be appreciated.
--
Paul Lew {oliveb,harvard,decvax}!gsg!lew (UUCP)
General Systems Group, 5 Manor Parkway, Salem, NH 03079 (603) 893-1000
-----------------------------
From: "Chris Calabrese[rs]" <cjc at ulysses.homer.nj.att.com>
Subject: Re: Getting the pathname from a FILE*.
Date: 9 Jul 88 20:15:24 GMT
Posted: Sat Jul 9 16:15:24 1988
To: unix-wizards at brl-sem.arpa
In article <651 at umb.umb.edu>, karl at umb.umb.edu (Karl Berry.) writes:
>
> The title says it all. Am I missing something obvious?
> The pathname doesn't seem to be a field in either the
> _iobuf or the structure returned by stat.
> ttyname or ctermid will give you the pathname
> of your terminal, but I want the pathname of an
> arbitrary FILE* I have fopen. Or am I missing some
> reason why this is not feasible?
If _you_ have fopened the file, then _you_ already know
what the name is (just have it sitting around in your program),
but if the file is inheritted(sp?) how do you even know it's a file???
If you rsh or rlogin, then the stdin to any process created on the
remote machine is a socket (mabee it's something similar, but you get
the idea), or a stream (ditto) for sysV.
Even if you don't have any kind of a network, then stdin could be
a pipe, which has no name for unnamed pipes. These are, infact, files
in most versions of sysV, but they have no names, just inodes.
--
Christopher Calabrese
AT&T Bell Laboratories
ulysses!cjc
-----------------------------
From: Bruce Robertson <bruce at stride.com>
Subject: Re: I/O Processors & Smart Screens
Date: 8 Jul 88 21:25:21 GMT
Posted: Fri Jul 8 17:25:21 1988
To: unix-wizards at brl-sem.arpa
In article <1568 at yendor.UUCP> gmg at yendor.UUCP (Gary Godfrey) writes:
>Has anybody ever heard of or worked with a UN*X system that has
>a "smart" screen handler in an I/O processor?
Visual Technology and XPI are jointly developing an X Server terminal,
which qualifies as a "smart screen handler". Also, I've worked with
systems which do some fairly high level things. For example, you can
download a set of characters and a column number, and the reading
process won't be unblocked until either a character in the downloaded
set is typed, or the column number is reached. This is useful for
programs like Emacs, to avoid context switches on every character
typed.
--
Bruce Robertson
MicroSage Computer Systems
bruce at stride.com
-----------------------------
From: Glenn Powell <gpowell at wizdom.uucp>
Subject: Re: Who dat?
Date: 9 Jul 88 15:58:58 GMT
Posted: Sat Jul 9 11:58:58 1988
To: unix-wizards at SEM.BRL.MIL
In article <2310 at rtech.rtech.com> daveb at rtech.com (Dave Brower) writes:
>On a single machine, we have a suid server process. A random client
>program wishes to connect and have the server perform some services. The
>server wants to know who the client is before doing anything. They might
>be communicating with sockets, fifos, msgs, or shared memory.
>
>How can the server find out who the client is, in a spoof-proof and
>secure way? On BSD, one can have the server ask the client to create a
>randomly-named file, and the server can see who the owner of the file
>is. On SV, this fails because the client can chown it to be anyone
>else. (The same is true of msgs and shm segments).
You are correct that any SysV user can chown() a file to anyone else. BUT,
if you require that the file be su'id they can't get away with it. The
chown() call will clear the suid bit when it changes the ownership and
nobody but root can set it on another person's file.
SO, have the client create a file with user execute+suid and no access
for other users. Make sure that they not only own the file; but, have
it set suid. It is important that there not be any access for other
users(for obvious reasons:).
This does work with named pipes(I just tried it) so it might work
with msgQ's and shared memory. But, I haven't tried that. Anybody
know for sure?
______ _________
/ ____/ /__ __/ Glenn Powell
/ /___ / / gpowell at wizdom.UUCP
/___ / / / uunet!wizdom!gpowell
____/ / __/ /__ Software Innovations Inc., Kansas City
/______/ /________/ Disclaimer: These opinions are MINE!
-----------------------------
From: Chris Torek <chris at mimsy.uucp>
Subject: Re: setsockopt -- SO_LINGER parameter?
Date: 11 Jul 88 13:20:55 GMT
Posted: Mon Jul 11 09:20:55 1988
To: unix-wizards at SEM.BRL.MIL
In article <4084 at saturn.ucsc.edu> koreth at ssyx.ucsc.edu (Steven Grimm) writes:
>Does anyone know what kind of value setsockopt() is expecting
>as a parameter to the SO_LINGER flag?
In 4.2BSD (and hence [old?] SunOS), an `int' which is the linger time.
To turn it off, you must use SO_DONTLINGER.
In 4.3BSD and later, a structure called `linger', defined in <socket.h>:
/*
* Structure used for manipulating linger option.
*/
struct linger {
int l_onoff; /* option on/off */
int l_linger; /* linger time */
};
--
In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7163)
Domain: chris at mimsy.umd.edu Path: uunet!mimsy!chris
-----------------------------
From: Michael Cohen <mike at bucasb.bu.edu>
Subject: BSD 4.3 F77 fortran query
Date: 11 Jul 88 14:02:07 GMT
Posted: Mon Jul 11 10:02:07 1988
To: unix-wizards at brl-sem.arpa
Does anybody know an easy way to popen a stream which can be passed
to Fortran and associated with a fortran Unit,. This is for a version
of BSD4.3 F77. What was already tried was a fortran open followed
by a dup2 using the fortran unit as a file descriptor as passed to C.
This doesn't seem to work. If anybody knows how to do this or
has the location of a routine which works this would be of interest.
-----------------------------
From: Karl Kleinpaste <karl at dinosaur.cis.ohio-state.edu>
Subject: Re: How do I get RCS?
Date: 11 Jul 88 14:25:13 GMT
Sender: news at tut.cis.ohio-state.edu
Posted: Mon Jul 11 10:25:13 1988
To: unix-wizards at brl-sem.arpa
chip at vector.UUCP writes:
>RCS is available via anonymous ftp on arthur.cs.purdue.edu. Grab
>pub/rcs/rcs.tar.Z.
Do any systems with anon uucp have this available?
Yup. Osu-cis, the usual instructions apply. Request
/u/public/rcs/rcs.tar.Z.
--Karl
-----------------------------
From: Chuck Karish <karish at denali.stanford.edu>
Subject: Re: BSD 4.3 F77 fortran query
Date: 11 Jul 88 14:35:23 GMT
Sender: news at labrea.stanford.edu
Keywords: popen Fortran Units
To: unix-wizards at SEM.BRL.MIL
In article <584632927.3487 at bucasb.bu.edu> mike at bucasb.bu.edu (Michael Cohen) writes:
>Does anybody know an easy way to popen a stream which can be passed
>to Fortran and associated with a fortran Unit,. This is for a version
>of BSD4.3 F77. What was already tried was a fortran open followed
>by a dup2 using the fortran unit as a file descriptor as passed to C.
There's no really portable way to do this, even among different
ports of 4.x. A colleague of mine wrote some code to do this
under 4.2. It uses low-level calls within the stdio library,
and depends on knowledge of exactly how a file descriptor and a
FORTRAN logical unit are implemented.
When I ported the code to Convex Unix (on a binary-only
system), I had to guess about the changes in the internal
interface. I got lucky, and it worked for a while. There's
nothing to keep Convex from streamlining their stdio library
and replacing or changing the name of one crucial interface, at
which point my code dies.
Chuck Karish ARPA: karish at denali.stanford.edu
BITNET: karish%denali at forsythe.stanford.edu
UUCP: {decvax,hplabs!hpda}!mindcrf!karish
USPS: 1825 California St. #5 Mountain View, CA 94041
-----------------------------
From: Brain in Neutral <bin at rhesus.primate.wisc.edu>
Subject: setgid and process group list
Date: 11 Jul 88 17:03:21 GMT
To: unix-wizards at SEM.BRL.MIL
Why does setgid screw up the list of groups that getgroups returns?
E.g., in a setuid root program:
ngroups = getgroups (NGROUPS, grparry)
print groups here...
setgid (somegid);
ngroups = getgroups (NGROUPS, grparry)
print groups here...
The group lists printed before and after the setgid are different,
at least under Ultrix 1.2 and 2.2. (Typically a group is missing after
the setgid).
Doing an initgroups after the setgid fixes things (I think). I notice
that BSD 4.3 login.c does this. Why is it necessary?
-----------------------------
From: Stan Hanks <stan at shell.com>
Subject: Re: Realtime OS's
Date: 11 Jul 88 17:18:29 GMT
Sender: usenet at shell.com
Posted: Mon Jul 11 13:18:29 1988
To: unix-wizards at SEM.BRL.MIL
Just a quick question, while we're talking about realtime OS:
what is the nominal interrupt dispatch time for Unix (time from the
the hardware interrupt line being yanked until your interrupt handler
is executing)?
If you post some info on this, please include
* the type of hardware you are using
* the which version of Unix you are using
* how you determined the time involved
Thanks. If you mail direct to me, I'll summarize to comp.os.misc.
Please post follow-ups to comp.os.misc.
Stan Hanks
Research Computer Scientist, (and Postmaster!)
Shell Development Company, Bellaire Research Center (713) 663-2385
...!{sun,psuvax,soma,rice,decwrl,cs.texas.edu,ihnp4}!shell!stan stan at rice.edu
Stan Hanks
Research Computer Scientist, (and Postmaster!)
Shell Development Company, Bellaire Research Center (713) 663-2385
..!{sun,psuvax,soma,rice,decwrl,ut-sally,ihnp4}!shell!stan stan at rice.edu
-----------------------------
From: Jeffery Siou <jeffs at wjvax.uucp>
Subject: Salvaging info on a streamer tape
Date: 11 Jul 88 19:09:56 GMT
Keywords: HELP
To: unix-wizards at SEM.BRL.MIL
Help! I need help/info regarding restoring information off a
streamer tape that is possibly scrozzeled.
Here's what happened....
I've been saving and adding information on a tape. I started out
at 'mt fsf 0' and every time I needed to add something I used 'mt fsf x' with
x being the next unused track(or block) and now I'm at 'mt fsf 18'. But some
how or some where when I tried to go to 'mt fsf 19' I got an 'I/O' error.
Then when I tried going to 'mt fsf 18' same error. I then rewound the tape' mt
rew' , All the info was correct from 'mt fsf 0' to 'mt fsf 8'. I then tried
'mt fsf 9', then i did a 'tar -tv', there was supposed to be 10 files
here, but 3 showed up and then I got a 'tar:read error'. I can not go
to anything above 'mt fsf 9' without getting an error. Now my question is
is there any way to salvage track 10 through 18? Again I can't get pass
'mt fsf 9' without getting an error.
Any help greatly appreciated...
Thanks in advance.
+---------------------------------------------------------------------------+
| jeffery siou |
| ...!{ !decwrl!qubix, ames!oliveb!tymix, pyramid}!wjvax!jeffs |
| |
| the above opinions are mine alone and not, in any way, those of WJ. |
| |
+---------------------------------------------------------------------------+
|
-----------------------------
From: Guy Harris <guy at gorodish.sun.com>
Subject: Re: BSD:sockets::SVID:(what?)
Date: 11 Jul 88 23:46:40 GMT
Sender: news at sun.uucp
To: unix-wizards at brl-sem.arpa
> This requires a small change in strwrite() and a distinquishable message from
> the tty module. However I would not call these `changes to the mechanism'.
*I* would call them that; adding a new message for these purposes *does* change
the mechanism.
In any case, you can't do it without changing "strread()" (which is what you
meant, not "strwrite()"), which is the real point here.
-----------------------------
From: "David M. Jones" <dmjones at athena.mit.edu>
Subject: undump
Date: 12 Jul 88 01:52:46 GMT
Sender: daemon at bloom-beacon.mit.edu
Posted: Mon Jul 11 21:52:46 1988
To: unix-wizards at brl-sem.arpa
Has anyone made undump work under System V? Acutally, we are running
IBM's AIX system on IBM RT's, but we have been led to believe that AIX
is a close derivative of System V. If anyone can point us toward the
correct version for our system, please let me know, preferably via
email.
Thanks in advance.
David M. Jones | "I tried to fake it,
ARPANET: dmjones at athena.mit.edu | I don't mind saying
UUCP: ...!mit-eddie!mit-athena!dmjones | I just can't make it."
-----------------------------
From: Naim Abdullah <naim at eecs.nwu.edu>
Subject: Is write(2) "atomic" ?
Date: 12 Jul 88 10:12:23 GMT
To: unix-wizards at SEM.BRL.MIL
Do UNIX semantics guarantee that write(2) calls will be "atomic" ?
Suppose, process A executes write(fd, "123", 3) and process B
executes write(fd, "456", 3) "concurrently". The file descriptor fd
is shared between them (the file was creat(2)'ed for writing by the
common parent of A and B). Does UNIX guarantee that the contents of
the descriptor will be "123456" or "456123" (depending on which of
A and B won the race) but never "124536" ? Does it make a difference
whether the descriptor is a pipe or a terminal or a disk file or a
tape drive or something else ?
Naim Abdullah
Dept. of EECS,
Northwestern University
Internet: naim at eecs.nwu.edu
Uucp: {oddjob, chinet, gargoyle}!nucsrl!naim
-----------------------------
From: Richard Caloggero <rich at eddie.mit.edu>
Subject: UUCP
Date: 12 Jul 88 15:46:57 GMT
Keywords: UUX, UUCICO, UUXQT, Apollo, Alliant
To: unix-wizards at SEM.BRL.MIL
Ok all you UUCP wizards, here's one for you! I know you're out
there ...
Until we obtain our ethernet software, we are using UUCP to do
file transfers between our Domain ring and our Alliant Fx8. Since
we are only communicating with one remote sise (actually, it ain't so
remote) and its our own machine, I have set things up a bit differently
than one might otherwise. Essentially, we only allow requests to be
initiated from the Apollo net, since we don't have enough resources to
dedicate an sio port on an Apollo. We define 2 operations: get_file
(get file(s) from Fx8), and send_file (Apollo -> Fx8). Both use the
uucp command to queue the specified set of files, then the program
creates a process on the Apollo node which contains the sio port used
to communicate with the Alliant, and starts uucico to do the transfer.
When uucico has successfuly started, {get,send}_file returns to the
shell. When uucico finishes, it uses write(1) to send a message to the
user who started it.
Now, the fun begins ... I want to be able to use uux to set up
remote command execution. The problem is, I can't seem to get the
command to start on the Alliant. I can set up the command with uux,
and uucico seems to send the correct files over to the Alliant, but
nothing happens on the Alliant, and I can't find the data which was
suposedly sent to the Alliant. I thought I found something when I
discovered that a core dump was being left in "/usr/spool/uucp/XTMP" on
the Alliant. I checked the permissions on this directory, and found
that world didn't have "w" rights. When I gave world "w" rights, the
core dump didn't happen, but then again, nothing else did either! Any
Idea as to what's gone wrong? ( I keep seeing references to a
directory called "/usr/spool/uucp/X.apolloX", but its not their on
either machine, and I can't seem to find anything in the manual that
would indicate a need for that directory. )
Thanx for your help. Please mail responses to me. I'll re-post a
summary if it seems appropriate.
--
-- Rich (rich at eddie.mit.edu).
The circle is open, but unbroken.
Merry meet, merry part,
and merry meet again.
-----------------------------
From: Richard Caloggero <rich at eddie.mit.edu>
Subject: File space allocation/deallocation under Unix (BSD4.x)
Date: 12 Jul 88 16:18:56 GMT
Keywords: Unix, BSD, File, Delete
To: unix-wizards at SEM.BRL.MIL
If you create a directory "d", then create a file "d/f" 1 megabyte
long (big), then "rm d/f", is it true that the space remains allocated
to directory "d" and cannot be garbage collected or otherwise reclaimed
until another file is created in directory "d" or "d" is deleted? If this is true, why?
What good does it do to delete files, say, in your home directory
(if, for example, disk resources are low and you system administrator
keeps hounding you to "clean up your directory")?
--
-- Rich (rich at eddie.mit.edu).
The circle is open, but unbroken.
Merry meet, merry part,
and merry meet again.
-----------------------------
From: Barry Shein <bzs at bu-cs.bu.edu>
Subject: Re: Is write(2) "atomic" ?
Date: 12 Jul 88 17:08:00 GMT
To: unix-wizards at brl-sem.arpa
>Do UNIX semantics guarantee that write(2) calls will be "atomic" ?
>
>Suppose, process A executes write(fd, "123", 3) and process B
>executes write(fd, "456", 3) "concurrently". The file descriptor fd
>is shared between them (the file was creat(2)'ed for writing by the
>common parent of A and B). Does UNIX guarantee that the contents of
>the descriptor will be "123456" or "456123" (depending on which of
>A and B won the race) but never "124536" ? Does it make a difference
>whether the descriptor is a pipe or a terminal or a disk file or a
>tape drive or something else ?
>
> Naim Abdullah
There's another possibility which is hard to illustrate but more like
write(fd,"123456",6) and write(fd,"ABC",3) -> "123ABC456" (what I'm
trying to say is atomic in units of some page size or block boundary)
and yet another (infra vide.)
I think the result is indeterminate (undefined), a few experiments
here on various systems came up with some even stranger results than
the above (as if the file was just overwritten, more like "123ABC" for
the [expected] 9-char example above.)
Note to experimenters: try it with one buffer (single write) very
large, like 100K or more. With smallish buffers things tend to
serialize.
Anyhow, that's what file/record locking is all about (NFS seemed to
strictly serialize things, however, I'm not sure if that's guaranteed
at all but I can see how it is more likely, just luck probably?)
-Barry Shein, Boston University
-----------------------------
From: Ron Natalie <ron at topaz.rutgers.edu>
Subject: Re: File space allocation/deallocation under Unix (BSD4.x)
Date: 12 Jul 88 17:11:56 GMT
Keywords: Unix, BSD, File, Delete
To: unix-wizards at brl-sem.arpa
Not true. When the last reference of a file goes a way, so does the space
it took up. What is probably misleading you is the fact that typical UNIX
directories themselves never get smaller when you delete files. First, you
need to realize what a UNIX directory contains. A UNIX directory only contains
file names (1-14 characters on most, variable length on BSD variants) and
pointers to the inode that that name refers to. An inode is a data structure
that contains the "essence" of the file, information such as the owner,
permissions, access times, and where on the disk the actual data contained
in the file is stored. When the last directory reference to that inode
is removed (there is a count kept in the inode), the inode and the data
it represents is freed. On traditional UNIX file systems, when you delete
a directory entry, the entry is just zeroed (actually only part is zeroed)
but it's slot is left in the directory for future reuse.
Consider that you create a directory with 1000 files in it. On a System V
file system a directory entry is 16 bytes long (14 characters for the name,
2 bytes for the inode number). The directory would grow to 16000 bytes long.
If you removed all the files, the entries would be zapped, but your directory
would still have 16000 bytes, representing 1000 free entries. You can compress
these "holes" by removing the directory (if it were empty) or copying the
directory over fresh by moving each file or subdirectory to a fresh directory
and discarding the old one. However, the data blocks actually consumed by the
files disappeared as soon as no one was still pointing to them.
-Ron
-----------------------------
From: John Owens <john at jetson.uucp>
Subject: Re: IBM speed
Date: 12 Jul 88 14:25:28 GMT
To: unix-wizards at SEM.BRL.MIL
In a previous article, Mark Williams <ccwilliams at wombat.dec.net.uq.oz> writes:
> Unix might be a bit faster if each user had his/her own one-cylinder
> filesystem.
Yes, especially without links or tree-structured directories.
If you want to organize your files under VM, you get several minidisks
(each an exact cylinder multiple) and put related files together....
I'll take the UNIX filesystem over that kind of speed any time. (I've
worked under VM, so I know what it's like.)
--
John Owens SMART HOUSE Limited Partnership
john at jetson.UUCP (old uucp) uunet!jetson!john
+1 301 249 6000 (internet) john%jetson.uucp at uunet.uu.net
-----------------------------
From: Richard Caloggero <rich at eddie.mit.edu>
Subject: UUCP
Date: 12 Jul 88 20:03:51 GMT
Keywords: UUCP, UUX, UUCICO ...
To: unix-wizards at SEM.BRL.MIL
Ok all you UUCP wizards, here's one for you! I know you're out
there ...
Until we obtain our ethernet software, we are using UUCP to do
file transfers between our Domain ring and our Alliant Fx8. Since
we are only communicating with one remote sise (actually, it ain't so
remote) and its our own machine, I have set things up a bit differently
than one might otherwise. Essentially, we only allow requests to be
initiated from the Apollo net, since we don't have enough resources to
dedicate an sio port on an Apollo. We define 2 operations: get_file
(get file(s) from Fx8), and send_file (Apollo -> Fx8). Both use the
uucp command to queue the specified set of files, then the program
creates a process on the Apollo node which contains the sio port used
to communicate with the Alliant, and starts uucico to do the transfer.
When uucico has successfuly started, {get,send}_file returns to the
shell. When uucico finishes, it uses write(1) to send a message to the
user who started it.
Now, the fun begins ... I want to be able to use uux to set up
remote command execution. The problem is, I can't seem to get the
command to start on the Alliant. I can set up the command with uux,
and uucico seems to send the correct files over to the Alliant, but
nothing happens on the Alliant, and I can't find the data which was
suposedly sent to the Alliant. I thought I found something when I
discovered that a core dump was being left in "/usr/spool/uucp/XTMP" on
the Alliant. I checked the permissions on this directory, and found
that world didn't have "w" rights. When I gave world "w" rights, the
core dump didn't happen, but then again, nothing else did either! Any
Idea as to what's gone wrong? ( I keep seeing references to a
directory called "/usr/spool/uucp/X.apolloX", but its not their on
either machine, and I can't seem to find anything in the manual that
would indicate a need for that directory. )
-- -- -- -- -- -- -- -- -- --
Ok, this is realy frustrating! As far as I can tell, everything
is being set up properly for remote command execution, but nothing
hapens. Also, I keep getting "xqt denied" messages is "LOGFILE". I
checked "L.cmds", and the "who" command is definately in that file.
The command "uux 'sun!who >/users/rich/who.sun'" doesn't execute on sun
(the Fx8). Help!!!!!!!!!!
-- -- -- -- -- -- -- -- -- --
Thanx for your help. Please mail responses to me. I'll re-post a
summary if it seems appropriate.
--
-- Rich (rich at eddie.mit.edu).
The circle is open, but unbroken.
Merry meet, merry part,
and merry meet again.
-----------------------------
From: Paul Gluckauf Haahr <haahr at phoenix.princeton.edu>
Subject: Re: Is write(2) "atomic" ?
Date: 12 Jul 88 18:29:20 GMT
To: unix-wizards at SEM.BRL.MIL
in article <11410005 at eecs.nwu.edu> naim at eecs.nwu.edu (Naim Abdullah) writes:
> Do UNIX semantics guarantee that write(2) calls will be "atomic" ?
in general, no. it depends on the implementation.
use some synchronization primitives or one byte writes only.
worse than just mixing up data, if two processes are pounding away
at a file, it data may be lost. (see below)
> Suppose, process A executes write(fd, "123", 3) and process B
> executes write(fd, "456", 3) "concurrently". The file descriptor fd
> is shared between them (the file was creat(2)'ed for writing by the
> common parent of A and B). Does UNIX guarantee that the contents of
> the descriptor will be "123456" or "456123" (depending on which of
> A and B won the race) but never "124536" ? Does it make a difference
> whether the descriptor is a pipe or a terminal or a disk file or a
> tape drive or something else ?
some special file types may implement atomic writes. notably, berkeley
sockets (at least under SunOS 3.5 and Ultrix 2.0) appear, empirically,
to be fully atomic. this is probably to support "reliable" protocols
like tcp/ip. (could someone who knows the tcp/ip protocol spec confirm
whether or not it requires atomic writes?) as a side effect, pipes
when implemented by socketpair(2) seem atomic.
i don't know about system v streams, but version 9 streams and pipes
are non atomic, and warn about this on their manual pages (along with a
comment that a fast reader and slow writers can simulate atomicity).
there is a shell archive at the end of this article containing two
programs i have used to test the atomicity of writes. the first,
write.c, creates two processes. each writes n strings of A or B to
standard output, with the length and number of strings set from the
command line. no synchronization is attempted. the original process
writes strings of A, the child writes strings of B. so, for example,
the output of "write 5 4" could be
AAAAABBBBBAAAAABBBBBAAAAAAAAAABBBBBBBBBB
(where 5 is the length of the writes, and 4 is the number of strings)
count.c reads the output of write.c and counts the number of each
character (sort of like uniq -c for characters instead of lines). the
shell script bufs translates the number of characters into number of
buffers, which will be fractional if there was a non-atomic write.
"count | bufs 5" for the previous data gives
5 1 A
5 1 B
5 1 A
5 1 B
10 2 A
10 2 B
where the problem comes in is larger buffers. using large enough
writes gives fractional numbers in the second column, on an nfs or
nd filesystem. on a sun, i have not been able to generate a partial
record, i.e. "124536" from the original article, with a local disk.
what i consider a more serious problem occurs much more frequently than
fractional writes. data gets dropped. this occurs with both local
and remote file systems. using a "write 8193 15" to a local (smd) disk
with an 8192 byte filesystem blocksize on a sun (similar results were
seen on a vax) gave
90123 11 A
8193 1 B
8193 1 A
8193 1 <<< empty
16386 2 A
114702 14 B
examining the file with od showed nul (0) characters in that area. it
takes fewer writes to get a similar result repeatedly with an nfs or nd
filesystem. what seems to be happening is that between the time one
process writes its data and when it updates the file pointer, the other
process gets scheduled to run.
to solve this problem, one would need to add locks or semaphores to
file table entries to guarantee exclusive access to the file pointers.
fortunately, the people who are doing (symmetric) multiprocessor unices
have to do this anyway.
paul haahr
princeton!haahr or haahr at princeton.edu
# to unbundle, sh this file
# bundled by haahr on dennis at Tue Jul 12 14:04:59 EDT 1988
# contents of bundle:
# write.c
# count.c
# bufs
echo write.c >&2
sed 's/^-//' > write.c <<'end of write.c'
-#include <stdio.h>
-
-#define atoi(s) (strtol((s), (char **) 0, 0))
-#define streq(s, t) (strcmp((s), (t)) == 0)
-
-extern char *malloc();
-extern long strtol();
-extern int strcmp();
-
-int main(argc, argv)
- int argc;
- char *argv[];
-{
- int pid, wpid, i, c, n, bufsize;
- char *buf;
-
- if (argc != 3) {
- fprintf(stderr, "usage: %s bufsize nwrites\n", argv[0]);
- exit(1);
- }
- bufsize = atoi(argv[1]);
- n = atoi(argv[2]);
-
- if ((pid = fork()) == -1) {
- perror("fork");
- exit(1);
- }
-
- if (pid == 0)
- c = 'B';
- else
- c = 'A';
- if ((buf = malloc(bufsize)) == NULL) {
- perror("malloc");
- exit(1);
- }
- for (i = 0; i < bufsize; i++)
- buf[i] = c;
-
- for (i = 0; i < n; i++)
- if (write(1, buf, bufsize) == -1) {
- perror("write");
- exit(1);
- }
-
- if (pid != 0)
- do
- if ((wpid = wait((int *) 0)) == -1) {
- perror("wait");
- exit(1);
- }
- while (wpid != pid);
-
- return 0;
-}
end of write.c
echo count.c >&2
sed 's/^-//' > count.c <<'end of count.c'
-#include <stdio.h>
-
-main()
-{
- int c, lastc, n = 0;
- do
- if ((c = getchar()) == lastc)
- n++;
- else {
- if (n > 0) {
- printf("%6d %c\n", n, lastc);
- }
- n = 1;
- lastc = c;
- }
- while (c != EOF);
-}
end of count.c
echo bufs >&2
sed 's/^-//' > bufs <<'end of bufs'
-#! /bin/sh
-n=$1
-shift
-awk 'NF > 0 { printf "%8d %-8.3g %s\n", $1, $1/'$n', $2 }' $*
end of bufs
chmod +x bufs
-----------------------------
From: David Brown <david at pyr.gatech.edu>
Subject: Anyone running 4.3+NFS from U of Wisc ?
Date: 12 Jul 88 21:07:49 GMT
Keywords: 4.3+NFS, UW, SunRPC3.9
To: unix-wizards at SEM.BRL.MIL
Has anyone running 4.3+NFS tried using the new versions of Sun's RPC
libraries? I'm having a few problems compiling existing programs
(can't find xdr_path), and would like to hear from anyone else who
has attempted using the new libraries. Please E-mail me.
Thanks,
David Brown
----------------------------------------------------------------------------
David Brown
Armstrong State College, Savannah, Georgia
uucp: ...!{akgua,allegra,amd,hplabs,ihnp4,seismo,ut-ngp}!gatech!gitpyr!david
ARPA: david at pyr.gatech.edu
-----------------------------
From: Chris Torek <chris at mimsy.uucp>
Subject: Re: File space allocation/deallocation under Unix (BSD4.x)
Date: 12 Jul 88 21:09:18 GMT
Keywords: Unix, BSD, File, Delete
To: unix-wizards at SEM.BRL.MIL
In article <9662 at eddie.MIT.EDU> rich at eddie.MIT.EDU (Richard Caloggero) writes:
> If you create a directory "d", then create a file "d/f" 1 megabyte
>long (big), then "rm d/f", is it true that the space remains allocated
>to directory "d" and cannot be garbage collected or otherwise reclaimed
>until another file is created in directory "d" or "d" is deleted?
The answer is `yes', but given the way you phrased the question, I think
you misunderstand what is going on. The space FOR THE NAME `f' remains
allocated. Since file names are typically very small, this makes little
difference (space is allocated in 512 byte or 1 kbyte units). On the
other hand, if a directory grows very large (many file names) and the
file names are subsequently removed, the space remains occupied, except
in 4.3BSD and later versions of BSD, where any new file creation or
renaming will truncate any unused blocks off the end. For this to
work, the remaining file names need to be near the front of the directory;
this can be arranged by renaming those files near the end:
# approximately,
# do
ls -f
mv last_name_shown x; mv x last_name_shown
# until last_name_shown does not move.
# the directory has now been compacted.
--
In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7163)
Domain: chris at mimsy.umd.edu Path: uunet!mimsy!chris
-----------------------------
From: Chris Torek <chris at mimsy.uucp>
Subject: Re: Is write(2) "atomic" ?
Date: 12 Jul 88 21:18:45 GMT
To: unix-wizards at brl-sem.arpa
In article <23801 at bu-cs.BU.EDU> bzs at bu-cs.BU.EDU (Barry Shein) writes:
>I think the result is indeterminate (undefined), a few experiments
>here on various systems came up with some even stranger results ....
It is certainly not well-defined. 4BSD makes writes to regular files
and block special files atomic by locking the inode across the write()
call. Character devices like terminals tend to be written atomically
only when the number of characters written fits in a cblock. Appending
by lseek/write (rather than FAPPEND) has a race between the lseek and
the write, and so forth.
--
In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7163)
Domain: chris at mimsy.umd.edu Path: uunet!mimsy!chris
-----------------------------
From: Chris Torek <chris at mimsy.uucp>
Subject: Re: Is write(2) "atomic" ?
Date: 12 Jul 88 23:32:41 GMT
To: unix-wizards at brl-sem.arpa
In article <3247 at phoenix.Princeton.EDU> haahr at phoenix.Princeton.EDU
(Paul Gluckauf Haahr) writes:
>worse than just mixing up data, if two processes are pounding away
>at a file, it data may be lost. (see below)
This is due to a bug in 4.2BSD (and probably others). Looking at
the 4.2BSD code, it seems to be that uio->uio_offset is not set
unless FAPPEND, instead relying on the value set before the ILOCK()
(all in /sys/sys/sys_inode.c ino_rw()). Two `simultaneous' writes
will cause the second to block in ILOCK(); meanwhile, fp->f_offset
will grow from the first writer, without uio->uio_offset growing
for the second writer when the first finally unlocks the inode.
>some special file types may implement atomic writes. notably, berkeley
>sockets (at least under SunOS 3.5 and Ultrix 2.0) appear, empirically,
>to be fully atomic. this is probably to support "reliable" protocols
>like tcp/ip.
I am not sure whether this is by design; it is due to the FIFO
behaviour of sleep() (see /sys/sys/uipc_socket.c sosend(), where more
is being sent than fits: if SS_NBIO nonblocking i/o, EWOULDBLOCK or
short write, else unlock socket buffer and sleep on it).
--
In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7163)
Domain: chris at mimsy.umd.edu Path: uunet!mimsy!chris
-----------------------------
From: 0732 <al at gtx.com>
Subject: where to do line editing?
Date: 12 Jul 88 15:18:28 GMT
To: unix-wizards at SEM.BRL.MIL
Ksh is a wonderful thing, no argument about that, and the best thing
about it is the history/command line editing feature. I can't help
wondering, though, if the shell is the best place for it (the editing
feature). Putting it in the kernel (e.g. stty vi, like stty cooked)
would give a uniform interface to all programs. I know, it is contrary
to the "UNIX Philosophy" to put user interface in the kernel, but that
would be sooo nice and the stty features have been creeping in that
direction anyway.
Alternatively, a library might be provided, like curses, that gave the
user a routine to fetch the next (history-edited) input line. This solution is
more of a pain to the programmer, since he has to look up the calling
sequence, explicitly program it in, etc. and the user can't depend on
it being done uniformly everywhere.
Comments, flames? Anybody done it, meant to do it, thought about it,
rejected it?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
( Alan Filipski, GTX Corp, 8836 N. 23rd Avenue, Phoenix, Arizona 85021, USA )
( {ihnp4,cbosgd,decvax,hplabs,amdahl,nsc}!sun!sunburn!gtx!al (602)870-1696 )
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----------------------------
From: Chuck Karish <karish at denali.stanford.edu>
Subject: Re: Is write(2) "atomic" ?
Date: 12 Jul 88 23:04:37 GMT
Sender: news at labrea.stanford.edu
To: unix-wizards at brl-sem.arpa
in article <11410005 at eecs.nwu.edu> naim at eecs.nwu.edu (Naim Abdullah) writes:
> Do UNIX semantics guarantee that write(2) calls will be "atomic" ?
If the write() is to a pipe or to a FIFO, and the request is for
[PIPE_BUF] bytes or fewer, the write() is guaranteed to be
atomic by the SVID and by the POSIX 1003.1 draft standard.
For other write()s, the behavior is undefined; you take your
chances. If you absolutely, positively need atomic write()s,
cram your I/O through a pipe or set up your own locking scheme.
Chuck Karish ARPA: karish at denali.stanford.edu
BITNET: karish%denali at forsythe.stanford.edu
UUCP: {decvax,hplabs!hpda}!mindcrf!karish
USPS: 1825 California St. #5 Mountain View, CA 94041
-----------------------------
From: Bob Pendleton <bpendlet at esunix.uucp>
Subject: Apologies
Date: 11 Jul 88 14:02:59 GMT
To: unix-wizards at SEM.BRL.MIL
Looking at my mail box this morning it is obvous that I made a serious
error in netiquette when I posted ile to comp.unix.wizards. My
apologies. I will not repeat this particular error.
I don't understand peoples objection to the copyright notice.
It is the same as the one in twm and very much like the ones in all X
code. All it really says is that you can't sell it and we hope you
won't sue us if you get hurt by using it. It seems reasonable to me.
Again, my apologies
Bob P.--
Bob Pendleton @ Evans & Sutherland
UUCP Address: {decvax,ucbvax,allegra}!decwrl!esunix!bpendlet
Alternate: utah-cs!utah-gr!uplherc!esunix!bpendlet
I am solely responsible for what I say.
-----------------------------
From: "Robert C. White Jr." <rwhite at nusdhub.uucp>
Subject: Re: Getting the pathname from a FILE*.
Date: 12 Jul 88 22:52:12 GMT
To: unix-wizards at SEM.BRL.MIL
in article <651 at umb.umb.edu>, karl at umb.umb.edu (Karl Berry.) says:
> The title says it all. Am I missing something obvious?
> The pathname doesn't seem to be a field in either the
> _iobuf or the structure returned by stat.
> ttyname or ctermid will give you the pathname
> of your terminal, but I want the pathname of an
> arbitrary FILE* I have fopen. Or am I missing some
> reason why this is not feasible?
At great risk of being wrong...
Since one i-node may have many file names I think that
the individual file name is disposed of as useless after
the object in question is opened. The closest I can even
picture you getting is to retreive the inode number.
All else would seem to be un-workable.
Rob.
Disclaimer: Then again, maby not.
-----------------------------
From: Doug Alan <nessus at wonko.mit.edu>
Subject: Input Line Editing
Date: 13 Jul 88 04:08:09 GMT
Sender: uucp at eddie.mit.edu
To: unix-wizards at brl-sem.arpa
In article <16456 at brl-adm.ARPA> rbj at nav.icst.nbs.gov (Root Boy Jim) writes:
> I suspect that the real place for line editing is either in the shell
> itsef (as in tcsh, ksh, (and brlsh?)) or in the kernel.
Putting line editing in the shell is wrong, because it should work in
all programs and be consistent. Putting it in the kernal is gross.
Thus, the right place to put it is precisely where Bob Pendleton wants
to put it -- in a process which gets input from the user and feeds
edited input to the user's other programs. If needed, mods to the
kernal and convention, however, should be made to make this as easy
and efficient as possible.
The same is true for output. This "termcap" stuff is a crock. There
should be a process which gets the output from programs, processes it,
and displays it. This process should be responsible for managing the
screen, and keeping track of what is on it for an entire login. Every
program having to clear the screen when it is started, so it knows
what state the screen is in, and having to have this big library
compiled in, is bogus. The ITS operating system knew better than this
decades ago. It had a decent virtual terminal system. So did MIT's
version of Tops-20 (it was in the kernal, however).
In fact, X currently does much of this. It should be made to be more
well-integrated, however. Every program should know about X, so no
program has to worry about managing the screen on its own. Perhaps X
should have input line editing added to it. Every window could have
an input editing sub-window, where the user would type and edit his
input. In fact, this is just what Apollo's do, now that I think of
it. Darn, they thought of it first!
There should also be a version of X that runs in a limitted form, but
supports the most basic features, on a normal, dumb terminal, so you
don't *have* to have a workstation to use programs that don't need
one.
|>oug /\lan
-----------------------------
End of UNIX-WIZARDS Digest
**************************
More information about the Comp.unix.wizards
mailing list