If you have read the article I wrote about distribution-friendly projects at LWN (part 1 and part 2, part 3), I tried to list some of the problems that distributions face when working with upstream projects.
One interesting thing that I did overlook, and I think was worth an article on its own, or even a book on its own, finding the time, is how to handle interaction between projects. All the power-users expect that the developers don’t try to reinvent the wheel every time, or if they do they do it for a bloody good reason. Unfortunately this is far from true, but beside this fact, when sharing code between projects it’s quite common that mistakes end up propagating in quite a huge area.
A very good example of this is the current state of Gentoo’s ~arch tree. At the moment there are quite a few things that might lead to build-failure because different projects are interleaved:
- GCC 4.3 changes will cause a lot of build failures in C++ software because the headers dependencies were cleaned up; similar changes happen every GCC minor release;
- autoconf 2.62 started catching bad errors in configure scripts, especially related to variable names; similar changes happen every autoconf release;
- libtool 2.2 stopped calling C++ and Fortran compilers check by default, so packages have to take care of what they do use;
- curl 7.18 is now stricter in the way options are set in its easy interface, requiring boolean options to be explicitly provided;
- Qt 4.4 is being split in multiple packages, and dependencies has thus to be fixed.
There are probably more problems thatn these, but these are probably the main ones. Unfortunately the solution to similar problems by a few projects is not to start a better symbiotic relationship between the various project, is to require the user to use a given version for their dependencies… which might be different from the version that the user is told to use for another project… or even worse, they import a copy of the library they use and use that.
Interestingly enough, it seems expat is really a good example of library imported all over, and less understandable than zlib (which is quite small on its own, although it has its share of security issues built-in). I’ve found a few before, and some of them are now fixed in the tree, but in the last two days I found at least four more. Two are in Python itself, returned from the death (yeah two of them, one in celementtree and one in pyexpat), one is – probably, not sure though – in Firefox, and thus in other Mozilla products, I suppose, and the last one is in xmlrpc-c, which has one of the worst build-systems I ever seen and thus makes it quite hard to fix the issue entirely.
Maybe one day we’ll have just one copy of expat in any system, and that would be shared by everybody… maybe.