Re: DLL loading

[Charles Wilson <cygwin at cwilson dot fastmail dot fm>]

Corinna Vinschen wrote:
> On May 24 11:47, Charles Wilson wrote:
>> I've always kinda thought that cygwin2.dll should use something like
>> 	http://alain.frisch.fr/flexdll.html
>> or
>> 	http://edll.sourceforge.net/
>> for cygwin DLLs.
> 
> That's nice stuff, but doesn't it solve a different problem?  Both
> implementations strive to overcome the problem of back references,
> not of control of placement of DLL s in memory.

Sortof.  edll implements its own loader, but the format of its "shared
libraries" are actually just .o objects (maybe partially linked objects?
not sure).

flexdll provides mechanism to simulate undefined symbols.

By combining these two ideas, cygwin exe's could implement the "don't
link statically against DLLs at all" part of your original #3 proposal
-- and allow to manage the loading of all "DLLs (or .o whatevers)" and
symbol resolution manually inside cygwin -- without changing the design
of all .exe's out there.  They still "think" they need symbol _foo from
dll X -- and as far as the whole compiling/linking process goes, and as
far as the running/loading process is concerned -- from the perspective
of the exe itself it gets the symbols it needs.

But...if one were inclined to go that direction, then I like Ryan's
alternate idea of using a dummy "stub" executable -- with no deps other
than cygwin1.dll -- to manage setting up the child's memory map and
loading (real, actual, pei-386) dlls, before handing off to the "real"
.exe.  That way, little changes except inside the cgywin1.dll (I assume
the "stub" exe would be a single, unique exe that cygwin1.dll uses to
manage fork() regardless of the exe of the actual forker or eventual
forkee()/exec().

'course, both depend on the ability to load a "DLL" at a specific
address, it's just in the edll case or a derivative of that scheme, a
"DLL" isn't a pei-386 ".dll" if you know what I mean.

> That doesn't make much sense, I think.  If you look into the latest
> /proc/$PID/maps output, you'll notice that even very big applications
> with lots of DLLs have all(*) Win32 DLLs in the space beyond 0x70000000.
> 
> (*) There's an exception, apisetschema.dll, which sometimes lands in
> the low memory area < 0x10000000.

Hmm...I haven't looked closely in the recent past, but I remember issues
with some win32 i18n library on Vista (locale.dll?) getting loaded in
low places where it didn't match child/parent.

> Usually we don't have a problem with Win32 DLLs.  It's the Cygwin DLLs
> which create the problem.  Part of the problem is ASLR, another part is
> that we have much more DLLs than a couple of years back.  Go just 5
> years back and count the number of libs only available as static libs,
> and compare that with the state of today.

Oh noes! We've made it too easy!

I'm sorry. Want me to go break libtool? :-)

> That sounds rather ugly.  If you do that you load all plugins even if
> you only need one.  That leads to the question why you don't just link
> all plugins statically against the application. 

Less change in the application, and still allows for end users to add
their own, non-aggregated extensions.  Eg. 1 aggregate + 10
user-unaggregated is better than 52 vendor-supplied + 10 user extension
DLLs.

In perl, I think you can get the "benefits" of this arrangement today,
simply by enabling shared extensions, but then listing explicitly all of
the vendor-supplied ones in the 'static' list, as Reini did with Cwd.dll
(and POSIX.dll?) in an early attempt at tackling this issue.

> You might also
> encounter surprising low-memory situations.

Well, yeah, I suppose.  If every perl process has to load the entire
9,717,782 byte 'Aggregate' DLL (*) instead of just the 3-400k of the
half-dozen it really needs...

(*) yes, I added them up...excluding cygperl5_10.dll.

> Apart from rebaseall, the only chance I still see mid-term is to find
> out if we can control the load address of a DLL somehow.

...and, if we (*) do, that it doesn't get broken by Windows8.

(*) for various unspecified values of "we"

--
Chuck