This is the mail archive of the cygwin@cygwin.com mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [avail for test] libtool-devel-20030121-1


Ralf, this is the last message in the thread you mentioned.

Charles Wilson said:
> You've shown that win32_libid() as it is currently can take up to 5
> seconds on a single "library", while most "libraries" can be
> identified in 0.3-0.6 seconds. But that doesn't address how much
> longer win32_libid() adds to the total link time when building a
> library.
>
> e.g. try this:
>
> You're linking libkdetext or somesuch. Halt the build right as the
> link command (/bin/sh ../libtool -mode=link ....) is printed. Save
> the command in a text file somewhere.
>
> ****REBOOT**** in order to clear your machine's memory/cache
> whatever. Run the libtool -mode=link command (saved earlier) by hand,
> with the current win32_libid() function -- and time it.
>
> ****REBOOT**** again. Then, time the link command again (but edit
> your prebuilt libtool script and replace win32_libid() with this:
>
> win32_libid () { echo="x86 archive import" }
>
> (e.g. a no-op.) This will tell us exactly how much extra time is
> added to a typical kde libtool-link command by the current definition
> of win32_libid() AND what percentage of the total link time that
> reflects.
>
> What I'm getting at is: is this 5 seconds for the absolute worst case
> dependency (compared to 0.3 for most of them) really that bad,
> compared to the total time involved in the link. How much does
> absolute correctness (e.g. no prone-to-failure shortcuts) cost us?

You never followed up on that. However,

I'm betting that we can get most of the speedup we need by adopting the function below. It's not quite as agressive as yours, but it makes fewer assumptions. Results using the following snippet:

time for fn in /usr/lib/* /usr/bin/* ; do
echo $fn : `cygwin_libid $fn`
done

are as follows. I ran the test twice with each script and took the faster score. Hopefully that accounts for cacheing issues.

old script:
real 4m59.249s
user 3m1.315s
sys 2m8.683s

Chuck's new script:
real 3m40.841s
user 2m1.821s
sys 1m36.813s

Ralf's proposal:
real 4m24.808s
user 2m38.784s
sys 1m52.503s

Ralf's proposal (with .exe fix):
real 3m17.747s
user 1m55.214s
sys 1m24.283s

Also, Ralf's could probably be sped up by using some of the 'head -n XXX' tricks in my version.

1801 files, 139MB.

Note that .exe's are marked as 'x86 DLL' in both scripts, because soon the patch which allows exe's to have export tables will go into binutils, and we ought to allow linking against them. And really, there is very little difference between .dlls and .exes.

The difference between this script and Ralf's proposal is that Ralf assumed that every file passed to the function was x86 and didn't check it. Also, any file named *.dll.a was assumed to be an import lib without checking. I believe that this loophole would tempt people to rename .a's as .dll.a's which would defeat the purpose of the entire system -- and would probably end up breaking tertiary programs. (**) I *don't* think folks will be tempted to rename .a's as .dll's, however. Also, Ralf's version (as proposed) marks .exe's as 'unknown' -- but that's easy to fix, and would have the side effect of improving Ralf's scores on my "benchmark".

The downside to my version is that this symlink:
libfoo.dll.a -> ../bin/cygfoo.dll
will be marked "unknown".(*) Ralf's version will mark it as 'x86 archive import' -- which is acceptable given how the result is used, but isn't really correct. The original script would correctly identify it as 'x86 DLL'.

(*) This could be fixed with some low-impact special case code (detect symlink, follow it, check the filename of the target), but I'll leave that as an excercise...

(**) The way libtool works, if cygA.dll depend on cygB.dll, then the flag '-lB' is automatically appended to the link command whenever I try to build C.exe which depends on cygA.dll. But, if somebody broke the rules which building A.dll because they only had a static libB.a (and renamed it to libB.dll.a to fool libtool), the A.dll will re-export the symobls in libB.a. And you don't want that. But perhaps this simply falls under the heading of "the user is allowed to shoot themselves in the foot(head)". But this is a mighty shiny and tempting bullet..."Hmm, I can't link my dll because I only have a static version of this dependency. Hmmm...let me rename it!"

--Chuck

P.S. I'll be out of contact for a while, so "discuss". :-)

Chuck's new version.
##############################################

#!/bin/sh
# changes:
# 0) use extension to identify .dlls (user can then shoot
# self in foot, but it is their own fault). Upside:
# much faster AND allows successful link against a
# *symlink* to a dll; original code did not.
# 1) only call objdump and nm once, and only when necessary
# 2) only need the first 3 lines of $OBJDUMP output; keep 10
# and throw away the rest.
# 3) don't call awk
# 4) only check to distinguish between static and import libs
# within the first 100 lines of $NM output. This seems safe.
# Normal import libs have an " I " symbol on line 5--8.
# libcygwin.a [a hybrid import/static lib] has it on line 32.
# only truly pathological cases would appear static for 100
# symbols before an import symbol appears...
# 5) 'file' is fast; use it first to decide that $1 is an archive
# before using $OBJDUMP to insure x86 and / or $NM to distinguish
# between static / import.
# 6) both .dll's and .exe's are marked as 'x86 DLL' -- because
# they are, and because .exe's will soon be able to have export
# tables, so we should allow linking to them.
win32_libid () {
win32_libid_type="unknown"
if echo $1 | grep -E "\.dll$|\.exe$" >/dev/null; then
win32_libid_type="x86 DLL"
else
if eval file $1 2>/dev/null | \
grep -E 'ar archive' >/dev/null; then
if eval $OBJDUMP -f $1 | head -n 10 2>/dev/null | \
grep -E 'file format pe-i386(.*architecture: i386)?' >/dev/null ; then
if eval $NM -f posix -A $1 | head -n 100 | grep " I " >/dev/null ; then
win32_libid_type="x86 archive import"
else
win32_libid_type="x86 archive static"
fi
fi
fi
echo $win32_libid_type
}

OBJDUMP=objdump
NM=nm
win32_libid $1
##############################################

Ralf's proposal
##############################################
#!/bin/sh
# changes:
# 1) only use nm for .a files
#
win32_libid () {
win32_libid_type="unknown"
if echo $1 | grep -E "\.dll$" >/dev/null; then
win32_libid_type="x86 DLL"
else
if echo $1 | grep -E "\.dll.a$" >/dev/null; then
win32_libid_type="x86 archive import"
else
if echo $1 | grep -E "\.a$" >/dev/null; then
if eval $NM -f posix -A $1 | grep " I " >/dev/null ; then
win32_libid_type="x86 archive import"
else
win32_libid_type="x86 archive static"
fi
fi
fi
fi
echo $win32_libid_type
}

OBJDUMP=objdump
NM=nm
win32_libid $1
##############################################




--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Bug reporting: http://cygwin.com/bugs.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]