This is the mail archive of the
cygwin-developers
mailing list for the Cygwin project.
Re: About the dll search algorithm of dlopen
- From: Michael Haubenwallner <michael dot haubenwallner at ssi-schaefer dot com>
- To: cygwin-developers at cygwin dot com
- Date: Wed, 1 Jun 2016 16:26:03 +0200
- Subject: Re: About the dll search algorithm of dlopen
- Authentication-results: sourceware.org; auth=none
- References: <574E835E dot 7090109 at ssi-schaefer dot com> <20160601110947 dot GE11431 at calimero dot vinschen dot de>
On 06/01/2016 01:09 PM, Corinna Vinschen wrote:
> On Jun 1 08:40, Michael Haubenwallner wrote:
>> Hi,
>>
>> two issues with dlopen here (I'm about to prepare patches):
>>
>> *) The algorithm to combine dll file name variants with the search path
>> entries needs to be reordered, as in:
>> - for each dll file name variant:
>> - for each search path:
>> + for each search path entry:
>> + for each dll file name variant:
>> check if useable
>
> Rationale? We only need to find one version of the file and there
> usually only is one. This is mainly for moduled systems like perl,
> python, etc.
While I indeed didn't face a problem here yet, the rationale behind is
that I need to install a large application that provides its own (portable)
package build & management system, including lots of packages probably installed
in the host (Cygwin) system as well, most likely in (slightly) different versions.
When these package management system now does provide a different dll naming
scheme than the host system, like stick to "liblib.dll" rather than "cyglib.dll",
and the application wants to dlopen its own "liblib.dll", currently the host's
"cyglib.dll" is loaded.
> However, if you can speed up the search process ignore the
> question...
This also depends on whether find_exec really is necessary here.
>> *) The directory of the current main executable should be searched
>> after LD_LIBRARY_PATH and before /usr/bin:/usr/lib.
>> And PATH should be searched before /usr/bin:/usr/lib as well.
>
> Checking the executable path and $PATH are Windows concepts. dlopen
> doesn't do that on POSIX systems and we're not doing that either.
Agreed, but POSIX also does have the concept of embedded RUNPATH,
which is completely missing in Cygwin as far as I can see.
However, there is one path name that can easily serve as minimal
"embedded RUNPATH" - the executable's directory.
This is where I do have a problem right now:
My own /application/bin/python2.7.exe is linked to libpython2.7.dll,
located in /application/bin. Now there is some python script that does
have some - strange enough - cygwin-conditional code that reads:
import _ctypes
_ctypes.dlopen("libpython%d.%d.dll" % sys.version_info[:2])
While this is questionable by itself, it really shouldn't load another
libpython2.7.dll than /application/bin/python2.7.exe has already loaded
just because dlopen using a different search algorithm than CreateProcess().
However, when dlopen is about to search some path list, I can imagine to
search the list of already loaded dlls first as well, but I'd prefer to
leave this up to LoadLibrary...
> Having said that, LoadLibrary will search the usual paths. After 2.5.2,
> we're leaving XP/2003 behind, and then we probably should tighten the
> search algorithm along the lines of
>
> AddDllDirectory ("/usr/bin");
> AddDllDirectory ("/usr/lib");
> [...]
> LoadLibraryEx (path, NULL, LOAD_LIBRARY_SEARCH_USER_DIRS
> | LOAD_LIBRARY_SEARCH_SYSTEM32);
/me fails to see how this does help with the missing embedded RUNPATH.
>> For consistency, IMO, when any searched path ends in either
>> x/bin or x/lib, we should search x/bin:x/lib.
>
> This might make sense, at least in the direction lib->bin.
Fine with me too.
Side note:
We also use some cl.exe/link.exe wrapper that supports LD_PRELOAD,
LD_LIBRARY_PATH, embedded RUNPATH, as well as lazy loading for both
LoadLibrary and CreateProcess: https://github.com/haubi/parity
Basically I'm wondering why Cygwin doesn't provide that (yet?)...
/haubi/