Re: cygport cross-compiling beta1

["Yaakov (Cygwin/X)" <yselkowitz at users dot sourceforge dot net>]

On Sat, 2010-07-24 at 00:26 -0400, Charles Wilson wrote:
> What does gentoo do with cross compilers and sysroots?  I know
> ebuild/emerge supports them; do they treat them strictly as support for
> cross compiles, or as installable images on the intended $host?  I
> suspect the latter...

Not AFAICS:

http://www.gentoo.org/proj/en/base/embedded/handbook/?part=1&chap=3#doc_chap2

> However, I take your point.  Looking at the fedora-mingw-cross
> distribution, it does appear that they intend to provide a cross
> compilation environment *only*; if you intend to package something for
> deployment to the $host (e.g. to win32), then that's a separate build
> process, using something other than rpm-build.

Correct, after all, RPM is, as it name indicates, a package manager, and
shipping RPMs for Windows software wouldn't be very helpful (although
7-Zip does open them).

OTOH, cygport is NOT a package manager, it is a build tool which creates
package tarballs, which are easily unpacked on other systems.  So I
wouldn't mind making a separate cygclass for that purpose, which would
also allow for $prefix customization.  In fact, it's basically ready,
but I'm not sure what to call it[1].

[1] http://cygwin.com/ml/cygwin-apps/2010-07/msg00174.html

> We could go that route as well; it just strikes me as a shame to "give
> up" since win32 is /right there/ underneath cygwin all along...
> 
> OTOH, even on fedora-x86, it'd be nice to be able to map $sysroot$prefix
> to some specific wine path (identity mount "inside" wine, as $prefix?),
> and "run" the binaries "in place".  That would let you, in some cases,
> run package test suites and the like.

For testing MinGW binaries, because they see only Windows paths, *either
way* you have a problem running in the sysroot IF $prefix et al are used
in the code at runtime (/mingw != C:\mingw) or if the tests pass
Cygwin-native paths to the executable.  If not, either because there is
no runtime data or modules to find, or because it is coded to be
relocatable (at least on Windows, which is not uncommon), then you
should still be able to test-run cross-built programs using sysroot
libraries by appending $sysroot$prefix/bin to path.  That might very
well be why Fedora moves libgcc and friends into the sysroot.

AFAICS the only thing you lose by configuring --prefix=$sysroot$prefix
is transferability between the sysroot and the host, IOW:

1) sysroot-configured libraries would not run on the host if runtime
data/module paths are hard-coded;

2) sysroot-configured ELF libraries would have an extra RPATH pointing
to the sysroot, which IIUC should be harmless (could this be mitigated
by patching libtool to not use $hardcode_libdir_flag_spec when $libdir
is in $sys_lib_search_path_spec?);

3) host native-built libraries would need $sysroot prepended to paths in
their foo-config/.la/.pc files (none of which should affect using host
libc's for bootstrapping) to be usable in the sysroot, and vice-versa
for sysroot-configured libs to be used on the host.

But if you're only using sysroots for libraries, and you treat
building-for-host and building-for-sysroot as two different things, then
none of that should really matter.

> See, my experience with "sys-roots" have been the "place where I install
> all the crap I will turn into the system image on embedded-platform-X".
>  And that includes $sysroot$prefix/bin/busybox, etc.

But if you're making a system image, then there will anyways be a lot of
stuff in the sysroot that you will not want on an embedded platform
(headers, .a and .la libs, pkg-config files, foo-config scripts, extra
docs, unused l10n, etc).  Gentoo's approach seems to be that you use a
different directory for a staging area, copying only what you need from
the sysroot (shared libs, essential runtime data) into there.

> Right, that's a better approach.  However, you'd probably still need to
> patch foo's foo-config script itself, so that it also accepts a
> --sysroot option, right?

Only if you configure foo with --prefix=$prefix, not with --prefix=
$sysroot$prefix.

> And, the configure.ac for packages that are clients of libfoo, may also
> need to be patched to support --with-foo-config=a-path if they don't
> already. But again, that would be acceptable upstream.

If foo-config is found with AC_(CHECK|PATH)_PROG, which it should be,
then you just pass FOO_CONFIG=$CROSS_SYSROOT$prefix/bin/foo-config to
configure.  A patch to do *that* would almost certainly be accepted and
would be simpler than --with-foo-config.

Which makes me think that perhaps I should provide definitions for
CROSS_SYSROOT_{BIN,INCLUDE,LIB}DIR to make the $prefix handling opaque.

> True, but foo-config + sysroot is broken. That's not ideal either.
> However: how many packages, that either provide or use foo-config
> scripts, are *WE* planning to build/support using the cygwin cross
> toolchains, immediately?

$ ls -1 /usr/bin/*[-_]config | wc -l
99

Now, seven of those are Cygwin service installation scripts, but OTOH
there are certainly more which I don't have installed right now.  Either
way, that leaves quite a number of libraries (not to mention other
software which depends on these libraries) which would be affected by
this.

> Hmm, hrrmmm, hummmm....so, I wanted to test your script to see what it
> would do.  The first one I picked was 'freetype-config'.  Guess what?
> 
> It already supports SYSROOT, by the simple expedient of prepending all
> output paths with the environment variable $SYSROOT (--libs sysroot
> support uses some additional logic):

Well, you happened to pick the *only* one of those 90-plus which do
support sysroots.

> What about using this, for your cross-sysrootize (note the backslash
> preceeding $CROSS_SYSROOT? (or, $SYSROOT)
> 
> for f in $@; do
>   sed -i -e "s|\([^t]\)$prefix|\1\$CROSS_SYSROOT$prefix|g" $f
> done

That would require defining CROSS_SYSROOT manually if cross-compiling
w/o cygport.

> Wrong? There is no "wrong" in this context; it's only software, and they
> each performed their function as designed.

Semantics; then the design is poor.

> And they explicitly request that any DLL using the name 'zlib1.dll' be
> exactly compatible with that one.  As it happens, a DLL built
> using zlib-1.2.x/win32/Makefile.gcc -- named 'zlib1.dll' -- isn't
> necessarily compatible with the official MSVC6.0-built "zlib1.dll",
> because that makefile doesn't use -mms-bitfields (among other reasons).

As for upstream's "request", its not necessarily their place to
determine our (or mingw.org's) library naming and packaging schemes.
Especially when their own Makefile apparently goes against their own
wishes.  (Dealing with years of upstream misconceptions wrt Cygwin has
led me to take upstream "advice" with a grain of salt.  Or sometimes a
shaker-full.)

> But even at mingw.org, their mingw-ish zlib package doesn't pretend to
> be "the" zlib1.dll. Instead, Keith builds it as "libz-1.dll" which is
> neither "zlib1.dll", nor "mgwz.dll".

Hmm, so I see.  I thought I had checked that.

> If I were to break our existing back-compat NOW, I'd probably attempt to
> be inline with the libraries distributed by mingw.org, and try to ensure
> that our version and theirs were built exactly the same. (And not
> attempt ABI compatibility with zlib.net's "zlib1.dll")

Agreed.

> The primary purpose, until now, of the mingw-{zlib,...gcrypt} libraries
> has been to support setup.exe and setup.exe alone.  I want to make the
> MINIMUM changes in these packages, until we are sure that each
> additional change doesn't break setup.exe.

And what does DLL naming have to do with setup.exe, which is
*statically* linked?

> Now, I don't OBJECT to doing so; in fact, I'll probably progress that
> way before the mingw-gcc packages are officially released.  But not
> today, and not by next Tuesday.  And now that I have an initial version
> that DOES allow setup.exe to build correctly, I'll change ONE THING at a
> time, testing setup.exe at each step, to ensure compatibility throughout.

Now that I nailed down the problem to how GCC 4.5 optimizes
autoload.c[2], both my packages and yours produce a working setup.exe.
The only question is mingw.org compatibility, which shouldn't be as
complicated as you're making it.

[2] http://cygwin.com/ml/cygwin-apps/2010-07/msg00188.html

> I think we are stuck with the same conflict as mingw.org, with our
> compiler: the compiler defaults to GNU bitfields, but the products are
> all provided with -mms-bitfields.
> 
> So...I think the place to fix this is NOT in the mingw-gcc's specs file,
> but instead in cygport's cross.cygclass[case:i686-pc-mingw32].
> 
> If mingw.org ever changes their default ABI to -mms-bitfields, then we
> would naturally follow suit.

OK, so I'll have to add it to CFLAGS et al, but it does mean that custom
Makefile-based packages need to make sure that $CFLAGS gets passed
along.  A bit messy, but I see no alternative.

> No, cygwin defaults to -shared-libgcc.  mingw uses -static-libgcc by
> default for C, shared for all other languages:
> 
> * Shared libgcc: If all modules are linked with -shared-libgcc,
>   exceptions can be thrown across DLL boundaries.  Note that this is
>   the default for all languages other than C. To disable this, use
>   -static-libgcc.

Now if (upstream) libtool would only accept these flags...


Yaakov