On the silliness of close() giving EDQUOT

[Robert Thurlow]

In <12045:Oct2604:56:3290 at kramden.acf.nyu.edu> brnstnd at kramden.acf.nyu.edu (Dan Bernstein) writes:

>In article <thurlow.656468483 at convex.convex.com> thurlow at convex.com (Robert Thurlow) writes:
>> In <9681:Oct2004:06:3090 at kramden.acf.nyu.edu> brnstnd at kramden.acf.nyu.edu (Dan Bernstein) writes:
>> >In article <thurlow.656303314 at convex.convex.com> thurlow at convex.com (Robert Thurlow) writes:

>> >> So how do you pick up errors on asynchronous writes?

>> >``What asynchronous writes?''

>> [The ones I get when I don't use O_SYNC.]

> [A discussion of async I/O that sounds like the VMS model.]

I think that the model you described would be a nicer way to do some
things; you'd be able to avoid a buffer copy, and a program would be
better able to tell exactly what was going on.  But I don't agree
that it impossible to get multiprocessing with the Unix behaviour.

>Another is truly asynchronous I/O: reads or writes that happen
>asynchronously, concurrently with your program. The right way to see the
>difference between synchronous and asynchronous I/O is to look at the
>lowest level of I/O programming. A synchronous write has the CPU taking
>time off from executing your program. It copies bytes of data, one by
>one, into an output port. An asynchronous write has a separate I/O
>processor doing the work. Your CPU takes only a moment to let the I/O
>processor know about a block of data; then it returns to computation.
>The CPU wouldn't run any faster if there were no I/O to do.

With a buffer cache, your CPU has to do a copy from user space into the
buffer cache, but it can then just submit an I/O request and get on
with other stuff.  If the I/O processor has access to the buffer cache
and the buffers are properly semaphored, the CPU can do what it wants
until it maps another I/O request onto the same block or it gets an
'I/O complete' interrupt.  In this case, a 'synchronous write' just
means your proc will remain asleep until after the I/O has been done.
The buffer copy kinda sucks, and I agree that the programmer can't use
your alternate I/O programming style, but I don't see that this makes
this "less asynchronous".

>> Our fsync, like Suns, ensures there are no pages in the VM system
>> marked as "dirty", and it does this by forcing and waiting for I/O
>> on each such page.  The I/O involves an NFS write, and any I/O errors
>> are detected.

>Are you sure? Suppose the remote side is a symbolic link to yet another
>NFS-mounted directory. Is the fsync() really propagated?

It doesn't go over the wire, which is another reason why we have to rely
on the actual NFS write operation being synchronous.  And the server won't
hand out filehandles that resolve to symlinks or accept them when given
as arguments to a write operation.  What kind of failure mode was on your
mind?

>This begs the real question: Why should I have to waste all that traffic
>on periodic fsync()s, when the traffic for timely EDQUOT detection would
>be a mere fraction of the amount? I can't afford to buffer everything
>and do just an fsync() before the final close().

I don't object to network traffic for EDQUOT detection a tenth as much
as adding state to my servers and junk to my protocol to allow a server
to ~dole out" a disk allocation to a client and be able to get it back
when it runs low.  If you want to build a bulletproof system that is as
immune to server and client failure as the current implementation of
NFS, I'd consider its merits, but if you ask me, we need a bug-free lock
manager/status monitor a lot more than we need something that just lets
you ignore error returns from close.

Rob T
--
Rob Thurlow, thurlow at convex.com or thurlow%convex.com at uxc.cso.uiuc.edu
----------------------------------------------------------------------
"This opinion was the only one available; I got here kind of late."