Re: ptmalloc2?

[Brian Ford <ford at vss dot fsi dot com>]

On Thu, 13 May 2004, Christopher Faylor wrote:

> On Thu, May 13, 2004 at 04:13:00PM -0500, Brian Ford wrote:
> >On Thu, 13 May 2004, Christopher Faylor wrote:
> >>Why does using ptmalloc buy you anything over the current scheme?
> >
> >Since I'm not very far into the port yet, I'm taking Doug Lea and
> >Wolfram Gloger's word for it, but ptmalloc was specifically designed
> >with SMP/multi-threading in mind to reduce contention.  That is the
> >assumed advantage.  I also assumed that if it was good enough for
> >glibc/Linux, then it might be good enough/better for us too.
> >
> >>You're asking if it's a good idea to use something without stating any
> >>benefits.
> >
> >I'm asking for obvious up front objections/encouragment and pointers to
> >gotchas, especially license wise since I'm not very familiar with that
> >issue (contributing ports of code you didn't write with compatable
> >licenses).
> >
> >I assumed people on this list might be familiar with different malloc
> >implimentations, and I thought I expressed my intent to attack a
> >certain specific (at least percieved) problem; thread contention.
> >
> >I'm confused about what was unclear.
>
> You sent email saying "We're seeing thread contention so I thought I'd
> contribute a port of ptmalloc2".

Thread contention in malloc, yes.

> The reader is left to do some inferring.

Given they are not familiar with the purpose of ptmalloc, maybe.

I'm still not sure what audience I'm targeting here.  Most of you are much
more familiar with the Linux and the open source world than I am.
Therefore, I thought you might also be more familiar with ptmalloc than I
am; didn't seem like an unreasonable assumption.

I've tried to write long, detailed messages here before and they generally
get no on-topic responses; ie:

http://www.cygwin.com/ml/cygwin-developers/2003-12/msg00020.html

So, I've adopted the short, assume-a-lot message style.  It seems to
provoke more responses, and I can always clarify when someone is
interested/confused.  The long ones appear to just take too much time to
read.

> One could infer that you think that ptmalloc2 is better at eliminating
> the thread contention that you think you have.

Yup, but I still have to prove it.  FWIW, Doug Lea seems to think so too:

If you are using malloc in a concurrent program, you would be far better
off obtaining ptmalloc, which is derived from a version of this malloc,
and is well-tuned for concurrent programs. (See http://www.malloc.de)

> One could surmise that you think that any decisions that glibc makes are
> ok for cygwin and so need no further technical review.

Nope.  I do assume that they are usually "well reasoned and insightful",
but discussion/review is always appropriate.

> In this message you mention a new name: "Wolfram Gloger", and we are
> left to conclude that he must be some kind of malloc genius whom we
> should hang our heads in shame for not knowing.

Sorry, maybe just the ptmalloc developer would have been better.

> If you have a need and want to make a major change to a fundamental part
> of cygwin, don't expect that everyone will drop everything and jump on
> over to glibc sources (or google) to look around to see what you're
> talking about.

I didn't.

> Make a technical argument for what you want

I plan to when I have data to back it up.

> and don't waste our time with assumptions about your perception of our
> familiarity with malloc implementations.

See comments above.

> Don't expect us to trust either you,

I know you don't :-).

> glibc developers,

Ok, I didn't know this was so strongly in question.

> or Wolfram Gloger to come to correct conclusions about what is required
> for cygwin.

Wouldn't think of it.

> In short, don't waste my time with vague assertions of a problem
> followed by vague suggestions of a fix,

I'd like not to waste my time with a port that would definitely not be
accepted because of some up front reason I didn't anticipate. See:

> >I'm asking for obvious up front objections/encouragment and pointers to
> >gotchas, especially license wise since I'm not very familiar with that
> >issue (contributing ports of code you didn't write with compatable
> >licenses).

Stress the "up front" and add "immediate with limited additional thought
or consideration".  Hence the:

> I thought I'd feel out the possibility of contributing a port of
> ptmalloc2

Feel out, to me at least, means initial reaction.  Please don't read in
more than I intended.

> and then jump to assumptions of the problems in integration

Those were not assumptions, but things I had already run into or
anticipated.  They were meant to provoke additional "initial reactions".

> before you've even proven that there is a problem

I've proven it internally, but it is difficult to relate this proof
here just yet.

> or that the proposed fix solves the problem.

Again, I didn't want to work on a predestined dead end solution.  That
seems reasonable to ask, no?

> Is this blunt enough for you?

Didn't know blunt was required, and sorry this was so long.

-- 
Brian Ford
Senior Realtime Software Engineer
VITAL - Visual Simulation Systems
FlightSafety International
the best safety device in any aircraft is a well-trained pilot...