The lives of distributions packagers are full of words that make them cringe – backport, regression, hotfix, custom patch, … – but there are two that can make your day truly shine: merged upstream.
When you’re maintaining a package for any kind of software, you have to mediate between the original upstream intentions and requests, and the distribution policy; in Gentoo, that policy includes respecting the user-chosen flags, especially since some of those are mandated by us and are needed to make sure that the software installed on the users’ systems is actually working as intended, but that’s just one of a long list of things you have to care about.
Most of the time, because of this, you have to end up patching the software itself: modifying the code so that it behaves like the distribution wants, even if that diverges from upstream behaviour, or in the best of cases, making it abide to both restrictions. Sometimes, you have to take a fix that has been already applied to the upstream repository, for instance in a development branch, and apply to the currently packaged version (backport). The least boring case is when users report a problem to you, that might or might not apply to other distributions, and you have to fix it anew.
When that’s the case, Gentoo’s guideline is to write patches so that they can be sent upstream (I also wrote about that — and I’m tempted to just re-publish my articles on my website rather than keep them there, especially given that the page is more broken each time I visit it). Unfortunately, some upstream are more difficult to work than others, plus sometimes the patches are really too Gentoo-specific that they make no sense to be sent upstream (for instance the S/Key patch for sudo, as it’s to support the Gentoo-specific port of OpenBSD’s S/Key support).
Now, as part of the Ruby team I’ve already written over and over about our need to patch a huge number of gems and other Ruby libraries, a lot of times simply to fix their Rakefile, less often to fix their tests… and in very rare occasions to hack or unhack the packages. Thankfully, these patches tend to be merged in quite quickly — I’m still not sure whether that’s due to the use of GitHub and of merge requests, or because releasing a Ruby gem is so quick and easy (which is a definite pro of RubyGems even for us packagers).
With other kind of software, the merge-and-release approach is not that common, but luckily there are exceptions; Robin Gareus has been terrific at merging my patches for liboauth and release a new version, which means that while I added it to the tree with three patches to be applied, you can get it now with no patch at all: vanilla 0.8.9!
Less quick to release, but that’s understandable considering the criticality of it, has been Linux-PAM; the newly released version, 1.1.2, which I added to the tree today, comes back to two patches, against the previous six of 1.1.1-r2. The remaining two are a bit tricky; one is something I’m keeping around since the 0.99 series, so a very long time ago, and is just a simple way for us to not build a bunch of test programs that we wouldn’t be using anyway; the other is a fix for the Berkeley DB detection in configure to work with the libraries are installed in Gentoo (with the prefix on the library name, but no prefix on the ELF symbols; we use versioning instead to avoid collisions between them). I’m now trying to find a compromise with Thorsten Kukuk (my upstream Linux-PAM contact) so that the patches can be applied, and we can stop patching it altogether.
Being able to ship packages that are not patched at all is important for many reasons, and at least one applies even to people who don’t use that package at all. Obviously, staying closer to upstream’s code is a positive thing because it means that you don’t risk that upstream counts your users out of support range (well, they can still do that if you are using non-default build options, but in general, it’s much easier for them to help you out if you don’t touch their code), but sending your own fixes to the original developers has another important result: the same fix is available to all the users of the program, not just those using your particular distribution. And finally, even people not using it will have a positive result, as long as they use Gentoo: less patch files in the tree means less files, and less overhead — and trust me, there is lots of overhead in the tree as it is right now, not sure if worse or better than what we had at the time I originally wrote that rundown, but it certainly needs some kind of proper solutions to be devised.
So I’m happy to thank for their merges Robin (liboauth), Thorsten (Linux-PAM), Karel (util-linux-ng), Cole (virt-inst), Thibault (fcron), Ludovic (ccid), … — the list goes a lot further, they are just the most recent upstreams I’ve exchanged email with!
On a side note; yes I picked up co-maintainership of bti with Greg; the reason is that I’ve been using it to dent the tinderbox results. I’ve also branched it upstream to cleanup a few things in the build system, and to implement one feature I’d very much like to use (--background); those fixes will come straight on 028 release, though, as they are not critical.
I always try to get things merged upstream with some degree of success. My work on Leptonica was heavy but the author was very grateful and hopefully we will be seeing these improvements in 1.67, when it can finally go into the tree. Upstream is becoming harder to define though. I know you’ve written about the proliferation of incompatible branches that the github ecosystem creates. I find that this problem is particularly prevalent around commercial games that are suddenly open-sourced. A whole bunch of coders, many of whom are not very familiar with open source, leap on it to make all their weird and wonderful changes without giving much thought to how these efforts could be coordinated and combined. Meanwhile, the original developers have washed their hands of it because they’re too busy making the sequel.
Glad to read some good news here 🙂
Another advantage is more extensive testing since more people use your patch, but the reduced maintenance when bumping should convince any maintainer to push patches upstream. Rewriting patches so they apply for a new version can become quite messy.
On a much smaller scale I get the same joy from merges 😉 (hint hint)I did re-write mine since you looked last I believe.
Yes yes I have a few different merges to apply to A-M, just hadn’t had very much time lately.