This is the mail archive of the cygwin-developers mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: DLL loading


On 22/05/2011 9:27 AM, Corinna Vinschen wrote:
On May 17 11:53, Ryan Johnson wrote:
On 17/05/2011 11:32 AM, Corinna Vinschen wrote:
On May 17 07:12, Ryan Johnson wrote:
On 17/05/2011 4:19 AM, Corinna Vinschen wrote:
What I'm observing is that even big apps like vim, emacs, octave don't
use addresses beyond 0x03000000.  Setting the heap to an address of
0x20000000 appears to be a rather big waste of memory.

So I'm planning to drop the bar to 0x08000000, which gives the heap
a potential extra memory of 384 Megs. and still leaves a confortable
cushion of 80 Megs for the OS.
On my machine, running 'emacs-X11 -nw', quite a bit of stuff appears
at 0x01????? (showing only allocation bases below for brevity):
[...]
Another bunch appears in the 0x19??????-0x1C?????? range (again,
allocation bases only):
19A80000-19C7B000 ---p 00000000 0000:0000 0
That's kind of annoying.  I wouldn't have believed that the OS DLLs
would take so much memory.  I'll stick to 0x20000000 for now.
annoying++
I'm still puzzled.  I used W7 32 bit and 2008 R2 64 bit for testing, and
even using really big apps like emacs-x11, xemacs, or the X server, I
can't reproduce that the OS uses memory beyond 0x04000000, on both
systems.  Are you sure that rebasing wouldn't fix this?

Along these lines there's another problem.

Apart from the fact that some of our Cygwin distro DLLs are still not
auto-based (the DLLs from libopenldap2_3_0-2.3.43-1, for instance),
what's really annoying is that Cygwin distro DLLs which have to be
rebased on the fly due to collisions with other Cygwin DLLs are rebased
to memory below 0x04000000, too.  Rebasing helps a lot, but there's
probably no way to avoid this entirely.  That's really bad.

Yes, we can't use a native fork mechanism, so we have to find another
way.  Three ideas come to mind:

- A more intelligent algorithm in rebase/rebaseall to place the various
   Cygwin distro DLLs so that they don't collide, perhaps together with a
   postinstall script which rebases automatically.  This is a short-term
   way to deal with the problem.

- Figure out if and how we can hook the Windows loader so that rebasing
   a DLL on the fly at load time can be influenced in terms of the start
   address.

- Stop linking against Cygwin DLLs other than the Cygwin DLL itself.
   Instead, provide our own loader.

Does anybody feel an affinity to have a look into one of the above?
Or, does anybody have another idea how to ease the pain?
I lean towards a variant of #3 -- always compile the user's code as a .dll, with the .exe containing just enough boilerplate code to dynamically load the app's .dll. That way, all the fork() code for taming dynamic loading can come into play without us having to write a full-custom loader or delve into Windows' hairy underbelly. It wouldn't be perfect but at least we'd have some control...

If folks are interested I can finish crafting the email full of details I had started to write... but I'll warn you now that I don't know enough about binutils and gcc (nor have time to learn it) to implement this plan in a transparent way.

A non-transparent prototype would be easy to hack together; I tried to do it with emacs, but it turns out emacs compiles a very basic binary, then fires up a *lot* of lisp runtime and writes that (initialized) lisp out as custom sections of a new .exe. Not a good one to mess with... most other static-dll-heavy apps should be easy to prototype this with, though.

Thoughts?
Ryan


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]