Tinderbox limits: blockers, collisions and conflicting requirements

For those who still don’t know that; my tinderbox is simply a properly configured container, based on LXC, that builds and installs, one by one, all the fifteen thousands packages available in Gentoo’s Portage tree — or at least those that are keyworded for either ~x86 or ~amd64 depending on the running instance.

This task is obviously time-consuming (which is why I keep shooting down the proposals of moving the tinderbox over to EC2: in my experience I/O is simply too slow for I/O-bound processes such as builds, and the fact that you pay for time doesn’t make it any better — S3 might be a better idea to store logs though, that is true), and it gets worse as the tree keeps growing. But that’s not half of it.

The most obnoxious issue relates to packages that provide (more or less) the same functionalities. Since Gentoo never implemented a generic and integrated system for alternatives – something I still envy Debian for – when two packages provide the same interface, or the same command, you end up with either choosing only one of them as the provider, or implementing a custom eselect module for each of them, neither of which is tremendously fun to consider. The third solution is the straight one, which is the fastest to implement but, from my point of view, the worst: you add blockers.

Blockers in Gentoo ebuilds simply mean that two packages cannot be installed at the same time in the same system, which is “perfect” when two packages both provide the same interface, such as jpeg and jpeg-turbo. But at the same time, this is bothersome when two packages install the same file… but with different meaning, which means you have to take one branch of the deptree and cull it out. And this is unfortunately common with too generic names for libraries, such as libnw that is either dev-games/libnw or sci-biology/newick-utils.

It might be of interest to note here that this became increasingly annoying with the various MTAs we have in Gentoo: ever since the mailwrapper idea was discarded, you could only get one of the different packages providing MTAs installed at the same time, as they all provided the sendmail command. This was relatively easy to deal with in the past because even if you added a new MTA to the tree, it would only have to register itself as virtual/mta and you had no blocker to update. With the introduction of the new-style virtual/mta package, the blocker needs to be fully-expanded in each and every MTA ebuild, which has become a bit of a burden. On the other hand, the new-style virtual also allowed some more fine-grained control, so that both msmtp and ssmtp are conflicting with other MTAs only if their mta USE flag is enabled.

Again, unfortunately the problem isn’t solved simply by blockers. Another problem happens when you have conflicting requests for USE flags. For instance, thunderbird-bin requires curl with the nss USE flag enabled, as that’s the Mozilla library implementing secure transports. At the same time, dev-libs/libcaldav wants curl with GnuTLS instead. And this is without adding collisions to the mess that make curl so much a pain in the back to deal with.

Some of the requirements are also pretty arbitrary, and usually stem from an overuse of “same flag” dependencies. So for instance if you got a program that can optionally use IPv6, it makes sense for it to depend on its backend library with IPv6 as well… but is the opposite true? If the program shouldn’t support IPv6, would the backend library still be able to have IPv6 support enabled? When the answer is yes, then you have a [ipv6?] dependency, while if they need to be kept in sync, you have a [ipv6=] dependency. Unfortunately I’ve seen it more than once that the latter form is used in places where the former was needed.

Hopefully, at some point, this could really be fixed once and for all.. even if it might end up requiring multiple compilation of the same library, for instance to have three different libcurl implementations with the different secure layer implementations, kinda like we have to build the same Ruby extension three or four times to install it for multiple implementations.

3 thoughts on “Tinderbox limits: blockers, collisions and conflicting requirements

  1. I’ve been running into the [flag=] that I’m almost positive should be [flag?] issue here myself, recently, on kde. I have to have USE=semantic-desktop kdelibs here, since I have (well, had, just switched to claws-mail) kmail installed, and that makes some sense. But why on earth does that mean I need dolphin, gwenview, plasma-workspace, etc, set USE=semantic-desktop as well? I’m almost positive all or at least most of those “leaf package” deps should be semantic-desktop? instead of the semantic-desktop= they actually have in the ebuilds currently.I plan on filing a bug on that if one’s not already filed, but haven’t yet, since it was only as I started investigating the deps in relation to my dissatisfaction with akonadified kmail that I realized what was going on, and I’ve been way too busy switching to claws-mail, since. But that’s done now, so…

    Like

  2. @Duncan: [semantic-desktop=] actually is (unfortunately) correct for most of KDE, as most things fail when kdelibs is built with +semantic-desktop and they are built with -semantic-desktop. It is unfortunate, but we had too many problems to be able to support anything other than +semantic-desktop everywhere or -semantic-desktop everywhere.

    Like

  3. I recently upgraded my cpu and decided to emerge -e world as things were a bit unstable. it is ~amd64 so *shrugs* I rm’d the /var/log/portage/* and checked logs along the way for ‘dereferencing pointer type punned will break strict aliasing rules’ since my gcc is 4.5.2 and there have been issues. I set in /etc/portage/env/category/pkgnameCFLAGS=”${CFLAGS} -fno-strict-aliasing” for those filesthe list is provided via ‘grep -H CFLAGS /etc/portage/env/*/*’/etc/portage/env/gnome-base/libgtop:CFLAGS=”${CFLAGS} -fno-strict-aliasing”/etc/portage/env/media-gfx/graphviz:CFLAGS=”${CFLAGS} -fno-strict-aliasing”/etc/portage/env/media-libs/libdvdnav:CFLAGS=”${CFLAGS} -fno-strict-aliasing”/etc/portage/env/media-libs/libdvdread:CFLAGS=”${CFLAGS} -fno-strict-aliasing”/etc/portage/env/media-sound/pulseaudio:CFLAGS=”${CFLAGS} -fno-strict-aliasing”/etc/portage/env/media-sound/sox:CFLAGS=”${CFLAGS} -fno-strict-aliasing”/etc/portage/env/net-irc/irssi:CFLAGS=”${CFLAGS} -fno-strict-aliasing”/etc/portage/env/net-wireless/kismet:CFLAGS=”${CFLAGS} -fno-strict-aliasing”/etc/portage/env/sys-boot/grub:CFLAGS=”${CFLAGS} -fno-strict-aliasing”/etc/portage/env/sys-boot/lilo:CFLAGS=”${CFLAGS} -fno-strict-aliasing”/etc/portage/env/x11-libs/pango:CFLAGS=”${CFLAGS} -fno-strict-aliasing”/etc/portage/env/x11-libs/vte:CFLAGS=”${CFLAGS} -fno-strict-aliasing”Now setting those for lilo and grub was unsucessful due to the ‘flag-o-matic’ filtering.Nevdertheless I managed to get it to work for lilo by moving that to an overlay and massaging the ebuild. ‘vte’ is rather insidious and all of these fall in the ‘do not report a bug to gentoo but instead fiile upstream’ categorydavid@random ~ $ equery depends vte * These packages depend on vte:dev-ruby/ruby-vte-0.90.8 (>=x11-libs/vte-0.12.1:0)dev-util/nemiver-0.8.1 (>=x11-libs/vte-0.12:0)gnome-base/gnome-2.32.1 (>=x11-libs/vte-0.26.2:0)net-misc/vinagre-2.30.3 (ssh ? >=x11-libs/vte-0.20:0)x11-terms/gnome-terminal-2.32.1 (>=x11-libs/vte-0.26.0:0)

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s