Reduced system set and dependency issues

I first proposed reducing the size of the system set – the minimal set of packages that form a Gentoo installation – back in 2008 with the explicit idea of making the dependency tree more accurate by adding the packages to the list of dependencies of those using them, rather than forcing them all in. While it is not going as far as I’d have liked at the time, finally Gentoo is moving in that direction and, a couple of weeks ago, a bunch of build-time packages were removed from the system set, including autotools themselves.

It is interesting to note that a few people complained that this was a moot point since they are among the first packages that you got to merge anyway, as they are build-time dependency of probably half the tree, so you can’t have a Gentoo system without them.. but reality is a bit happier when you look at it. On a “normal” Gentoo system, indeed, you can’t get rid of autotools easily: by default, since build-time dependencies are preserved on the system even after build has completed, they wouldn’t even be removed — on the other hand, they wouldn’t get upgraded until a package needs to be built that use them, which is still a nice side effect. Where this matters most is on systems built with binary packages, which is what I do with my home router and on the two vservers that host this blog and xine’s bugzilla so I can make use of the higher-end hardware of Yamato.

By removing autotools from the system set and instead expecting them to be listed as dependencies (as it happens when using autotools.eclass), the binpkg-based systems can be set up without installing autotools at all, in most cases. Unfortunately; that is not always the case. The problem lies with the libltdl library, which is libtool’s wrapper to the dlopen() interface — the so-called dynamic runtime linking, which is implemented in most modern operating systems, but with various, incompatible interfaces. This library is provided by the sys-devel/libtool package, and is used at runtime by some packages such as PulseAudio, OpenSC and ImageMagick, making libtool a runtime dependency of those packages, and not simply a build-time one. Unfortunately, since libtool itself relies mostly on autoconf and automake, this also adds to the runtime dependency tree those packages, causing the system to be “dirty” again.

Luckily, it appears that libltdl is falling out of preference, and is used by a very limited set of packages nowadays. The most obnoxious to avoid is ImageMagick, especially on a server. I don’t remember whether GraphicsMagick allows you to forgo libltdl altogether when not using dynamic plug-ins, I think it should but I wouldn’t bet on it.

More obnoxious are probably the few failures caused by not depending on less-commonly-used tools such as flex and bison (or their grandparents lex and yacc). While I did some work at the time I proposed the system set reduction to identify packages that lacked a dependency over flex, new packages are added, old packages are reworked, and we probably have a number of packages that lack such dependencies. It is not only bothersome for the users, who might find a failure due to a package not being installed when it should have been; it is also very annoying for me when running the tinderbox because I can’t get the complete list of reverse-dependencies to test a new version of the package (it happened before that I needed to test the reverse-dependencies of flex, it wasn’t nice).

This begs a question: why isn’t my tinderbox catching these problems? The answer is actually already out there since the end of the same year: my tinderbox is not focusing non minimal-dependencies. That is, it runs with all the packages installed at once, which means it checks for collisions and can identify (some) automagic dependencies, but it rarely can tell if a dependency is missing in the list. Patrick used to run such a tinderbox setup, but I don’t know if he’s still doing so. It sure would be handy to see what broke with the recent changes.