using malloc and brk: something to watch out for

[Dave Martindale]

In article <49 at houligan.UUCP> dave at smaug.UUCP (Dave Cornutt) writes:
>Keywords: malloc, brk
>Summary: beware of stdio doing malloc behind your back
>
>If you want to use brk() or sbrk() to do memory allocation, there is
>something you have to keep in mind: the standard I/O library does
>mallocs to allocate buffers for files that it handles.
>
> .....
>
>The moral of the story is this: (1) once you have done a brk or sbrk,
>don't do any mallocs, and (2) if you use stdio and brk/sbrk in a
>program together, make sure that all files being handled by stdio have
>buffers allocated to them prior to doing any brk/sbrk by either
>doing I/O on them (which forces a buffer to be allocated), or by
>calling setbuf() to explicitly allocate a buffer (alternatively,
>you can set it to unbuffered, although this usually hurts performance).
>Don't forget about stdin/out/err.

The problem with doing things this way is 1) it is vulnerable being broken
by the next person who works on the code and doesn't understand the
interaction and 2) it may break when you reload it with a new version of
some library that does a malloc where it formerly used a static buffer.

Much better is to use a single consistent memory allocation scheme.
If you are currently using sbrk just to get a large chunk of memory, just
malloc the chunk instead, and then there will be no conflict.

If you really need the flexibility of having a massive chunk of memory
that you can dynamically grow and shrink, then you do need to use brk;
in that case you can write your own malloc that is compatible with
whatever private memory allocation scheme you need to use, and stdio
will then use your malloc.

Either of these is a lot more robust than having two different memory
allocation strategies that your carefully arrange not to interfere with
each other, you think.

>Now for the flame: There should be some way to confine malloc to a certain
>heap space by setting an upper limit on how high it can allocate memory.
>This way, the programmer could use malloc and still have clear memory
>above the top of the heap space.

I disagree.  Malloc is designed to be the single memory manager for the
vast majority of C programs (and it's more portable than sbrk/brk).
For that it works adequately, and I don't think it should be more
complicated in order to handle an unusual situation like yours.  There
will always be some strange situation it will be unable to handle no
matter how it is written, and you can always supply your own version.