Re: gcc4: next release

[Charles Wilson <cygwin at cwilson dot fastmail dot fm>]

On 7/7/2010 8:39 AM, Corinna Vinschen wrote:
> On Jul  7 08:08, Charles Wilson wrote:
>> I hope I have summed up the various competing proposals fairly, and that
>> this edition of my patented War and Peace emails helps move the
>> discussion along to a conclusion.
> 
> Ok, I'm sufficiently confused now.  So, here are a few questions.
> 
> - Why do we need two different mingw's again?  What are the merits of
>   having mingw32 *and* mingw64-32?

mingw32 and mingw64-32 are different.

This is true whether the mingw64-32 in question is provided by (a) "gcc
-m32" where the gcc in question is a 64bit-by-default multilib gcc
derived from mingw64's code, (b) "gcc" where the gcc in question is a
32bit-by-default multilib gcc derived from mingw64's code, or (c) a
32bit-only version of gcc derived from mingw64's code.

However, the answer to your question differs depending on whether (a),
(b), or (c) -- or some combination -- is true.

For (a), Yaakov believes that "multilib makes sense for a 64bit compiler
on a 32bit platform". The mingw64 guys have their own reasons for
wanting to provide a multilib gcc, rather than simply two separate
"single-lib" (?) compilers.  Given those reasons for having "mingw64-32"
-- kinda for "free" along with mingw64's unique 64bit contribution, the
question becomes: why also provide mingw32?

That boils down to the various differences between the two: the
licensing and provenance issues connected with the two projects' w32api,
the fact that mingw.org is dw2 and (should, eventually) support a usable
gcj/java and Ada implementation, while mingw64 does not appear to
support either language -- and its sjlj exception model is slower than
dw2, painfully so in java according to reports.  OTOH, sjlj is required
if you want to unwind exceptions through foreign frames; that is, pass a
function to a w32api call as a callback, and allow that callback
function to throw. You just can't do that with dw2; your app will crash.

So, for (a) -- we get mingw64-32 for "free", but probably want also
mingw32.org because it has certain difference that make it more
appropriate for some tasks/languages/(legal concerns?).

(b) Well, if we're going to have mingw64-32 in some form, we might as
well treat it like a "normal" cross compiler where you do
--host=the-32bit-variant-I-want instead of --host=64bit-mingw-compiler +
CFLAGS="-m32".  However, that argument holds for both a "single-lib"
32bit gcc based on mingw64, and for this option (b) multilib beast.
Consensus, however, appears to be turning against a default-to-32-bit,
*multilib capable*, mingw64 gcc, no matter what ELSE is part of the
cygwin milieu.

(c) Again, if the 64bit mingw64 gcc is multilib, then the mingw64-32
stuff is going to be present.  We might as well also have a "dedicated"
mingw64 gcc focused specifically on that subtarget, since
--host=i686-w64-mingw32 would confuse fewer of our users than
--host=x86_64-w64-mingw32 CFLAGS="-m32". ("What? I don't have a 64bit
machine! Whine, groan, grumble...")

The argument for ALSO providing a mingw.org 32bit compiler, when we
already have a mingw64 32bit compiler, presented above still hold: the
mingw.org compiler has certain differences that may make it a more
appropriate choice for certain tasks/languages/(legal concerns?).

---
Now, if we overrule the mingw64 people (and Yaakov), and say "No, thou
shalt not provide a multilib compiler, not even your 64bit compiler",
then there is less reason to worry about mingw64-32.

In that case, we could have a 64bit (only) compiler based on mingw64,
and a 32bit (only) compiler based on mingw.org.

The downside of this is, that's not what the mingw64 *people* appear to
want to give us. We usually grant a large amount of deference to the
people actually doing the work.

Also, it'd be nice to be able to compare and contrast the two cross
compilers in operation on cygwin...how else could we test whether cygwin
is compilable with mingw64's w32api (*and* startup objects, since those
appear to be different? Does that matter for cygwin-1.dll?)

> - Along the same lines, what are the advantages of having two sets of
>   Windows headers and three sets of libs and DLLs?

I think I may have hit most of that, above, but:

* only mingw64's version is 64bit clean. If you want the 64bit compiler,
  you have to have the 64bit-compatible mingw-runtime and w32api (ne'
  "crt" and "headers"). So that's one of the copies. And, if the 64bit
  compiler is multilib, then you have to also have the "32bit" flavor
  of mingw64's libs/DLLs/headers anyway -- and that's two copies.

* mingw64's w32api files are more complete, even for 32bit. So...that's
  a definite plus.  But, the choice of using mingw64's w32api and
  mingw-runtime (e.g. "crt" and "headers") also implies a different
  ABI: sjlj vs. dw2, and different startup objects, etc.  AND it
  implies -- at least for the moment -- a lack of support for mingw
  gcj and Ada.  So there are pluses and minuses here.

* mingw.org: Well, why bother with these files any longer, even if
  we use a "mingw.org" gcc (that is, --target=i686-pc-mingw32, rather
  than --target=i686-w64-mingw32)? Well, the mingw.org gcc will
  probably need the mingw.org version of mingw-runtime, even if you
  do swap out the w32api bits for mingw64's version. (b) you'll need
  multiple installations -- or symlinks -- anyway, because all of these
  cross compiler variants' files will live under a per-$target area
  (e.g. a sysroot).  So, you end up with a copy of w32api specific to
  each compiler anyway, regardless of what source that w32api was
  obtained from.

  Also, as a consequence of mingw64's w32api files more complete state
  -- and how they got that way -- the legal provenance of those files
  may be suspect depending on how paranoid your company's lawyers are.
  Providing a fallback to mingw.org's "safer"(?) alternative may be
  important.

When you add all that up, you get three (*) copies of the windows
headers and three sets of libs/DLLs.

(*) three, not two (unless you play games with symlinks) -- because I
think you end up with two identical copies of the mingw64 headers so
accommodate the different --sysroots of the cross compilers (one for
mingw64 and one for mingw64-32).

> - Where are the differences of w32api from mingw32 and w32api from
>   the mingw64 project?  One of them is apparently that the mingw64
>   headers are 64 bit clean.  What else?

* mingw64 more complete, even for 32bit
* provenance and legal status of mingw64 w32api may cause concerns
  for some people
* they're structured differently. winsup/w32api and winsup/mingw are
  distinct entities with specific purposes.  mingw64's repo splits
  things differently:

    mingw-w64-crt/lib32/     .def files for w32api 32bit
    mingw-w64-crt/lib64/     .def files for w32api 64bit
    mingw-w64-crt/crt/       source code for startup objects
    mingw-w64-crt/*/         other mingw-runtime-like libraries

    mingw-w64-headers/crt/       headers for (parts of) mingw-runtime
    mingw-w64-headers/include/   headers for mostly w32api, but
                                 other items mixes in?
    mingw-w64-headers/*/         various other headers

  what's interesting is that 'mingw-w64-crt' and 'mingw-w64-headers'
  are completely separate projects, with their own top-level configure
  script and automake goodness.

> - Last but not least, why don't the projects merge and only keep
>   one set of w32api headers and libs?  After all, they have the
>   same development target.  A merge would help everyone, afaics.

I assume you're talking about merging ONLY the w32api components, and
not the mingw-runtime elements, nor the two projects themselves.

In my understanding, the issue is mostly the legal concerns with w32api.
 The technical concerns, e.g. the way mingw64 has intermingled what
mingw.org and cygwin call "w32api" and "mingw-runtime" makes things
difficult for a true merger to occur, especially as I don't think you're
suggesting that also the mingw-runtime bits be merged between the
projects, too.

Now, IF the legal concerns were addressed (*), THEN if it were deemed
possible for "mingw.org" to adopt, willy nilly, anything they found
relevant in mingw64's w32api headers and .defs -- well, they could
probably set up a mechanism to keep "matching" files synchronized.  But
the structure *around* those matching files...difficult, very difficult.
 Which means having a single repository controlling all of what cygwin
calls "w32api" probably won't happen, unless we (cygwin) and mingw.org
somehow convince mingw64 to *undo* the intermingling they deliberately
performed.

(*) see discussion, here:
http://cygwin.com/ml/cygwin-apps/2010-07/msg00081.html

Thinking ahead, suppose the current concerns about "hey, where did you
get that function definition? You didn't copy it from the Windows SDK
did you?" were addressed (or, is mingw64's "clean room" technique
satisfactory for mingw.org, as well as cygwin/Red Hat, and consistently
applies across ALL of mingw64's w32api?)

At that point, the status would be as follows: we'd have mingw64's
w32api-ish code under public-domain|ZPL, but mingw32's w32api under the
'Anders Norlander Custom License' (call it "ANCL"). Well, mingw32 could
grab anything of mingw64's public domain, and import it into their ANCL
code (provided they did so in jurisdictions where "public domain" is
recognized; IANAL, etc etc).

But...mingw64 would not be able to take any new advances from mingw32
and pretend they were in the public domain (as they'd be explicitly
marked ANCL, NOT pd!) -- so the cooperation would be pretty one way.
Unless each contributor provided the patch explicitly to both projects
under different terms; but how can you 'disclaim copyright and declare
public domain' when handing the code to person A, but 'assert copyright
and license under ANCL' (or some other copyright-dependent license),
when giving it to person B.  Either you own the code or you do not.  We
are not Schrodinger's Programmers.)

This means, for equitable sharing in the future, either mingw32 must
change its copyright status of w32api from ANCL to 'public domain|ZPL'
to match mingw64 (and NOT to MIT/X as they are currently contemplating),
or mingw64 must change the licensing structure of THEIR code from
'public domain|ZPL' to ANCL or whatever mingw32.org ends up with.  And
never mind the fact that Anders Norlander still holds the copyright to a
lot of that code, so...mingw32.org has to get his permission to do
anything, and he's dropped off the net.  Earnie Boyd is currently
attempting to contact him via his previous employer, but...

> Having said that, I can't see why keeping mingw32 would have any real
> advantage, except that it's part of the winsup build tree, so we get the
> headers and libs for free when building Cygwin.

Well, it has long been contemplated to move w32api and mingw-runtime out
of winsup/, but no one has ever pulled the trigger.  If that were to
happen, then even this "advantage" would go away.

But, there is still the whole "provenance of mingw64's improvements"
question to consider, and whether Red Hat wants cygwin to be dependent
on the answer to that question.

> Other than that,
> switching to mingw64 only seem to have advantages, given that it gives
> us the first gcc targeting 64 bit Windows as well, so there's a real
> chance to create a 64 bit Cygwin in the next 10 years.  Can anybody
> enlighten me?

Yep, it does seem to be a necessary first step.

> Oh, and, talking about /opt or /usr, I'd prefer the above /usr/mingw*
> sysroot idea.  However, I don't like the idea in the least to keep
> two different versions of w32api around.  It's one target, so we should
> have one set of headers only.  Right?  Wrong?  None of that?

The fedora structure is actually

   /usr/sysroot/$target-shortname

(e.g. "sysroot" is indeed an adjective, as you used it in your
statement, but it also appears literally in the pathname)

I like it (/usr/sysroot/mingw*/) too: but we will most likely have MORE
than two copies of (at least part of) w32api.

w32api, in cygwin terms, consist both of the headers AND the import libs.

The 64bit import libs are *different* from the 32bit ones -- but they
have the same names. So, they have to live in different locations.  You
might be able to share the headers betwen the 32bit mingw64 compiler,
and the 64bit one, assuming they are not different -mNN modes of the
same executable but...once you have two different and distinct versions
of "libkernel.a", why be concerned about two identical copies of winbase.h?


I guess we could always structure it using symlinks, but...that seems to
make it more difficult to manage, rather than less.


But here's the kicker: if w32api headers and (32bit versions of) implibs
"live" in /usr/sysroot/mingw32/{include,lib} -- ALONG with the headers
and libs for, say, mingw-xz, mingw-zlib, and any other mingw32 stuff an
end-user might compile and install on their machine using the cross
compiler...

Where should *cygwin's* compiler look for w32api headers and libs?
Certainly not there!  (Did I mention that the mingw64 "w32api" headers,
"mingw-runtime" headers, are all simply installed in include? There is
no provision for a 'w32api/' or 'mingw/' subdirectory -- mingw is mingw
is mingw, and all the headers are mixed together.) So, you can't tell
cygwin-gcc to look in /usr/sysroot/mingw32/include/w32api for the w32api
headers, nor in /usr/sysroot/mingw32/lib/w32api for the w32api implibs...

This tells me that *cygwin* will need its own copy of the w32api headers
and implibs -- whether that copy is derived from mingw.org's version or
some selected subset of what mingw64 calls 'crt' and some selected
subset of what mingw64 calls 'headers'.

That's part of the reason JonY and I were trying to nail down a naming
convention for the various packages, so that -- with appropriate
documentation in the FAQ or cygwin docs -- one could tell at a glance
just what the heck all these different -w32api- (or -headers-) packages
contained.

--
Chuck