Virtual memory behavior under SUNOS 4.0.3 (Some more thoughts)

[simon at corona.att.com]

Feedback from my previous posting has yielded the following tidbits:

Disk blocks (either direct I/O or over NFS) are cached in physical memory
in SUNOS 4.0.3. Physical memory pages consumed by disk buffering are NOT
considered to be free by vmstat. The text pages of processes are cached in
physical memory and reclaimed where possible - something which is true in
4.3 BSD as well. Physical memory consumed by text page buffering is NOT
considered free by vmstat.

I verified disk buffering behavior experimentally on an idle system:

    rebooted 24M system -> Large amount of memory shown to be free by vmstat
    dd'd small file to /dev/null -> forced text pages of "dd" to be loaded
    ran vmstat to determine free physical memory.
    dd'd unreferenced version of vmunix to /dev/null
    ran vmstat again. (Excess physical memory still reported - no page outs )
    The difference was 760K or 778240 bytes. The size of the file was 789309.

Since the only memory draw-down could be due to disk buffering action, the
test implies that all disk i.o. pages are buffered in physical memory when
possible. In addition, one can cause program pages to be freed and/or
paged out by reading / writing large files concurrently. This suggests
that disk buffering competes on a similar footing as other virtual memory
operations for physical memory. 

Other UNIX boxes partition the demands of disk buffering into a reserved
chunk of physical memory. My suspicion is that there are both benefits and
drawbacks to the approach SUN has taken of allowing flexible allocations
of physical memory to disk buffering and program paging.

I have yet to make any tests to see how data pages are returned to the
free list after process death. It is widely acknowleged the vmstat is
deficent in reporting virtual memory consumption (active virtual memory).

What prompted my original posting was a suspicion that the virtual memory
subsystem of SUNOS 4.0.3 on the sparcstation was not performing as
optimally as it could. I would appreciate hearing from anyone who is
familiar with the innards of the SUN kernel and its design trade offs.

Marc Simon ATT Bell Labs
simon at corona.ATT.COM 201-615-2535