Re: "kernel" heap idea

[Chris Faylor <cgf at cygnus dot com>]

On Tue, Jan 04, 2000 at 06:40:35PM -0500, DJ Delorie wrote:
>Many unixes have a set of functions that function like
>malloc/realloc/free, but use the kernel's memory heap instead of the
>application's heap.  With Cygwin, such a feature could be used to
>manage a chunk of shared memory instead of hard-coding items into the
>existing shared memory area.  This would especially be useful for
>cases where the number of items is variable (like delqueue).

Somebody submitted a patch to manage shared memory like malloc
a while ago.  It would be nice to be able to do this.

>While reading up on how to do this under Win32, I did notice
>two shortcomings:
>
>* NT doesn't guarantee that a given virtual memory range will be
>  available for the next cygwin program, so the shared memory region
>  may not be able to be mapped in the same place for each invocation
>  of the dll.  I don't think this will be much of a problem for most
>  apps, as the cygwin dll gets to initialize itself pretty early, but
>  we may want a way to let the user pick a different address (perhaps
>  through the tunable parameter idea I posted earlier) to avoid
>  conflict with other allocations.

We rely on the fact that memory is available at the same location for
fork so if this breaks we have serious problems anyway.

Also, as long as the memory doesn't contain internal references to
itself, you could get by with relocation.

>* Win32 doesn't let you expand a shared memory region once you've
>  created it, so if we run out of memory in that region, we're out.
>  Again, this may be a case where a tunable parameter is the right way
>  to deal with this, and we'd need to be careful with functions that
>  use the shared area to make sure they return the right errors if it
>  runs out.

It doesn't let you expand a region but you can use the same mechanism
as we do for the heap where you set aside a region but don't really
allocate it.  So, we could set aside 256MB or so but only use 16MB,
expanding as we need it.

If we are going to be inventing some kind of malloc-like allocation
in cygwin, it would be nice (tm) to also have a completely separate
malloc heap for cygwin itself.  linux certainly doesn't share it's
memory allocation for things like fd's with programs and I don't think
that cygwin should do so either.

I think that we could generalize the malloc in newlib to handle multiple
heaps.  We could then use it to provide a separate kernel heap and for
cygwin shared memory.

cgf