New libc malloc patch

[Daniel Eischen]

On Sun, 11 Dec 2005, Jason Evans wrote:

> On Dec 11, 2005, at 9:58 PM, Daniel Eischen wrote:
> > On Sun, 11 Dec 2005, Jason Evans wrote:
> >> I have been contemplating creating a separate spinlock API that
> >> doesn't require the threads library to track the spinlocks across
> >> fork.  This would (if I understand correctly) remove the current
> >> static spinlock limitations.
> >
> > What about using pthread_atfork()?
>
> Aren't there potential ordering issues for that?  It seems to me that
> the malloc pre-fork code would need to be run after any other pre-
> fork functions, in order to avoid potential deadlock, and that the
> malloc post-fork code would need to be run before any other post-fork
> functions, again to avoid potential deadlock.
>
> After looking at the spinlock code some more, it's no longer clear to
> me why the thread/thr_spinlock.c code uses a static array for
> spinlocks.  It seems to me that it would work fine to allow the
> program to provide space for a spinlock and manually initialize it.
> This would remove the limitation on the number of spinlocks.

We really want to deprecate the use of spinlocks in libc, so it
is just a band-aid until we change our mutex types to be full
structs instead of pointers to them that have to get allocated.
When that happens, the malloc implementation should be changed
to use mutexes since they will not have to be allocated, and, for
the uncontested case, be just an instruction or two (very much
like umtx, possibly the same).

When the imminent symbol versioning hits the tree, it will
be much easier to make the mutex change since it will not
affect everything in the world that uses mutexes, CVs, etc.
We just need to decide on a layout for them so all 3 thread
libraries agree, at least on size and on what FOO_INITIALIZERs
do.

> >> As for supporting recursive spinlocks, I doubt that the overhead
> >> would be acceptable in general.  If I could get rid of the need for
> >> the one recursive lock in malloc.c, I certainly would. =)
> >
> > Why do we need a recursive mutex?  Can you not restructure the
> > code so that it is not needed?
>
> There is an internal arena that the malloc code uses for allocating
> internal data structures.  In some cases, the internal arena has to
> recursively allocate.  If there were no object caching, it might be
> possible to pre-allocate, such that recursion never happens, but
> given the object caching, it's difficult to reason about precisely
> what will occur internally for a single malloc/free operation.  There
> are some other possibilities, but nothing I've thought of so far is
> simple or elegant.

Well, just lock around the external functions and remove all locking
from the internal and recursive functions.  Can't all recursion
be replaced with loops anyways ;-)

> Fixing this would make all locking in the malloc code a bit cheaper,
> which is why it continues to bother me.

--
DE

_______________________________________________
freebsd-current@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@..."

[Johan Bucht]

Daniel Eischen wrote:

>On Sun, 11 Dec 2005, Jason Evans wrote:
>  
>
>>>>As for supporting recursive spinlocks, I doubt that the overhead
>>>>would be acceptable in general.  If I could get rid of the need for
>>>>the one recursive lock in malloc.c, I certainly would. =)
>>>>        
>>>>
>>>Why do we need a recursive mutex?  Can you not restructure the
>>>code so that it is not needed?
>>>      
>>>
>>There is an internal arena that the malloc code uses for allocating
>>internal data structures.  In some cases, the internal arena has to
>>recursively allocate.  If there were no object caching, it might be
>>possible to pre-allocate, such that recursion never happens, but
>>given the object caching, it's difficult to reason about precisely
>>what will occur internally for a single malloc/free operation.  There
>>are some other possibilities, but nothing I've thought of so far is
>>simple or elegant.
>>    
>>
>
>Well, just lock around the external functions and remove all locking
>from the internal and recursive functions.  Can't all recursion
>be replaced with loops anyways ;-)
>
>  
>

Simple description of the issues following.

First ideal case, no recursive locking needed:

1. Lock the local arena to allocate memory
2. If we need to allocate internal data structures lock the base arena
2.1. Allocate memory from the base arena
2.2. Unlock base arena
3. Unlock local arena

The ideal case assumes that the base arena have everything it needs and
don't have to set up internal data structures to handle the allocations.

Actual case:

0. From malloc(3) friends get a local arena and pass that into the
internal functions
1. Lock the supplied arena
2. If we need to allocate internal data structure goto step 1, using the
base arena instead of one of the local arenas
2.1 Allocate memory from base arena
2.2 Unlock base arena
3. Unlock arena

This means that we can lock the base arena recursivly, should the
allocation of memory from the base arena need to allocate memory.
Solving this would need some special handling for the base arena (avoid
using the same interface as the other arenas or statically allocate the
base arena).

/Johan Bucht
_______________________________________________
freebsd-current@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@..."