This is the mail archive of the
cygwin-apps@cygwin.com
mailing list for the Cygwin project.
Re: patches to vendor source trees - discussion
Robert Collins wrote:
>>How about a script? Call it say pristine-src or something like that.
>>You would give it a package-version for the parameter. Then the script
>>would apply the patch and if successful remove the patch file. You
>>could even give it some options to allow for a backup of the current src
>>directory first.
>>
>
> Sure. But this is all predicated on the patch being in the source dir.
> (And someone needs to write the script).
>
> Chuck, I haven't heard from you on this bar the initial comments - and
> given the number of packages you maintain....
>
> Also, please note: I'm not suggesting that anyone has to repackage
> existing stuff... only that new packages, and new releases of existing
> packages should follow the guidelines we establish. I think anything
> else would be madness.
You've stated several times that you're looking only at "baby steps" -- but
I think baby steps are counterproductive here. Imagine a sequence of 5
baby step changes in the src packaging. There are three (bad) choices:
1) after each baby step, force all maintainers to update their src packages
to the new standard. This is, as you say, "madness".
2) Enforce that after each baby step, new packages and updated packages
must obey whatever new standard is in place at that time. this will lead
to a HUGE hodgepodge of src packages, each obeying different "standards" as
the standard evolves over these baby steps.
3) don't force anybody to change. You end up with the worst of (2) coupled
with no real "standard" at all.
I'm not opposed to doing away with CYGWIN-PATCHES, or putting the patch
file as a sibling to the package-src directory, or whatever -- I just think
these changes, these baby-steps, will prove counterproductive in the long
run. A data point: I wanted to update my 10 dll-ized library packages so
that they would accept the -DALL_STATIC flag as opposed to requiring
various incantations of -DZLIB_STATIC -DPNG_STATIC ....
This took what? six months? The update cycle for dependent packages is
VERY long -- if you want to minimize risk. Thus, the likelihood of (2)
occuring is very high, IMO. If we're gonna restructure the src packaging,
let's define the BULK of it now, and tweak later -- instead of tweak now
and radically restructure later, or tweak now, tweak later, and continue
tweaking forever. *Now* is when the new super-spiffy setup.exe is coming
out. Big changes are in the wind, why not this, too?
Of course, I'm not saying enforce immediate switchover of all packages NOW
-- just define the new standard, however radical, and then switch each
package over when it is updated in the normal course of events.
As much as Chris doesn't like it, there ARE advantages to the RPM style.
Here's my take (I'll use zlib as my example):
/usr/src/cygwin/SRC
*unpacked* sources go here. Packages that do not support
VPATH or for some reason must be built within their src
directory are also compiled here.
/usr/src/cygwin/BUILD
"normal" packages are compiled here.
/usr/src/cygwin/ORIG
.tgz's and primary sources (patch files and the like)
go here. "ORIG" is not an ideal name, but it's shorter
than "PRIMARY_SOURCES"...
/usr/src/cygwin/SCRIPTS
(think SPECS)
/usr/src/cygwin/INST
(I've never liked RPM's habit of using /var/tmp/<pkg>/
for temporary installs}
/usr/src/cygwin/PACKAGES
This is where the newly-created .tar.bz2's and
-src.tar.bz2's end up.
/usr/src/cygwin/DOCS
cygwin-specific READMEs and stuff.
So, zlib-1.1.3-9-src.tar.bz2 contains:
/usr/src/cygwin/ORIG/zlib/zlib-113.tgz
(pristine tarball, in whatever format the primary
developers provide it. No tarball renaming.
/etc/setup/zlib-src.list.gz will take care of
removing it at the proper time. BTW, *DOES* setup.exe
worry about files installed from -src packages?)
/usr/src/cygwin/ORIG/zlib/zlib-1.1.3-9.patch
contains diffs and additional files to make zlib
build properly. Does NOT create the cgywin-specific
README (that goes in DOCS). Does NOT create the
zlib-postinstall script.
/usr/src/cygwin/ORIG/zlib/zlib-1.1.3-9.postinstall
/usr/src/cygwin/ORIG/zlib/*
other patches and semi-standard source files (like the
lossless jpg patch)
/usr/src/cygwin/DOCS/zlib/zlib-1.1.3.README
Goes here, not in the .patch or inside the tarball.
Why? because these cygwin-specific READMEs are not
usually intended for submission back to the upstream
maintainers. So, keep 'em out of the .patch; makes it
easier for the cgywin maintainer to organize the patches
for upstream submission...
Also, this README no longer contains build instructions,
(except "run /usr/src/cygwin/SCRIPTS/zlib-build. Here
are the allowed options:")
/usr/src/cygwin/SCRIPTS/zlib-build
poor man's rpm+spec. A shell script that unpacks the
source (into SRC), applies the appropriate patches,
creates a build dir under BUILD, and creates an inst
dir under INST. Configures, builds, installs (into
INST/<pkg>/usr/* -- this includes copying
ORIG/zlib*postinstall into
INST/<pkg>/etc/postinstall/zlib.sh),
and creates packages. Note that THIS file doesn't go
into a "SCRIPTS/zlib" subdir -- while anything installed
or created in the other CAPITAL directories does go into
a zlib subdir. The reason? There will never be more
than ONE build script for any given package.
And that's (pretty much) it.
Why do I want zlib-build to handle so much? Why do I want so much stuff in
ORIG, and NOT created within the unpacked src dir by .patch?
It's very difficult to add a file to a package, AND munge the Makefile's to
install it. Okay, it's not *technically* difficult -- for non-autotool
packages you just *do* it. For autotool packages, you have to add the new
file to the Makefile.am, then re-run automake and autoconf -- which leads
to a LOT of changes in a LOT of the files == BIG patch.
But in both cases, these changes are NOT ultimately destined for upstream
inclusion. They are cygwin-only, and will never make it into the "real"
package. IMO, .patch should be limited to code changes that are
upstream-compatible. Or at least to those changes that aren't frufru.
Heck, in most of cgf's packages, the README (if it exists) is just
hand-copied into his inst dir immediately before tar'ring -- and is often
forgotten (binutils? <g>) Adding a file to a package The Right Way(tm) is
just plain HARD. Why enforce extra work? (BTW, I now officially hate the
"put patch file into CYGWIN-PATCHES" idea -- even though it was mine,
originally. I'm always forgetting to remove it before creating the new
patch, or forgetting to put it back in before creating the src tarball. Ugh.)
Using a build script (zlib-build) allows us to have multiple patches,
applied in the proper order. We don't need to "revert" to pristine source,
'cause that's what we're distributing. AND, this sort of structure is
easily translated to dpkg or rpm style tools, if we ever go there. (Plus,
scripts -- once they are properly written -- don't make mistakes, or forget
to include the patch in the src tarball, etc)
My proposal also minimizes the hurdle that new porters must overcome -- IF
we (I?) provide a "sample" build script. Any other plan requires porters
to do a LOT of work. For instance, I don't know how **I** would get
readline or ncurses to build with a single patch and one "configure; make;
make install" run. Currently, I do *two* make; make install runs -- and in
the readline case, there's a secondary patch applied before the second
(DLL-building) make; make install. Worse, I don't see this changing until
libtool supports the new dll-building/auto-import stuff in binutils (or
whatever gcc profile-based alternative Robert comes up with)
The point? Simpler src packaging standards don't allow for these complex
builds -- and in my experience, ports of new packages seem to always go
thru a stage where building is complicated. Then later, as the port is
refined (*) the build becomes simpler.
(*)more stuff accepted by upstream maintainers, better autotool integration
-- not to mention ld improvements (past) and libtool improvements (future).
Do we want a simple src packaging standard -- that serves as a disincentive
for new package ports, because it doesn't allow for complex builds? "No,
I'm sorry -- you can't submit that package until you can get it to build
using this src packaging standard." "But I can't -- I need to wait for this
change to autoconf and that change to binutils -- THEN I can get it to
build 'simply'" "Sorry, you must table your package. Go work on autoconf
and binutils, then come back"
too much delayed gratification. If I had had to WAIT until libtool was
working nicely so that "./configure --build-shared --build-static" worked
for ncurses on cygwin, we'd still be waiting.
Closer to home: if we go with an extrememly simple src packaging standard,
I won't be able to update readline for a LONG time (unless I get a
"waiver") -- because I won't be able to support the current ugly build
procedure of readline under a new -src standard if it isn't
flexible/powerful enough. (Oh, yeah -- and then there's the multiple
packages from a single source problem as in ncurses/libncurses: Can't be
done without a well-written build script. In the simpler src packaging
standards, where would IT go? Right now it's in CYGWIN-PATCHES, but only
for lack of a better place)
The danger is in making the "complex" -src standard too difficult. Well,
since my proposal is "put these files in those places, and use a build
script to do all the work" and "oh BTW, here's a sample build script,
adjust to suit your project" I think it isn't too difficult.
And it is quite similar to the structure used by rpm (dpkg) so it should
facilitate moving to that at some point in the future, if we decide to do so.
Then the answer to "How do I build cgywin from CVS" becomes
cd /usr/src/cygwin/SRC
cvs co winsup
mv src cygwin-cvs
cp cygwin-cvs/winsup/maint/cygwin-cvs-build /usr/src/cygwin/SCRIPTS/
cd /usr/src/cygwin/SCRIPTS
./cygwin-cvs-build doitall
and poof! /usr/src/cygwin/PACKAGES/cgywin-<date>.tar.bz2 and
cygwin-<date>-src.tar.bz2. And cygwin-<data>-src.tar.bz2, naturally,
follows the -src standard, and contains its own cygwin-build script
(slightly different from the cgywin-cvs-build script, so this requires two
similar build scripts in winsup/maint. Think about it, you'll realize why)
--Chuck