This Time Self-Hosted
dark mode light mode Search

Ever-present automagic dependencies

I’ve now redirected the tinderbox to run the build with OpenSSL 1.0.0a, to assess how much of the tree still needs to be ported before moving to the new OpenSSL release; this is another tricky update if you don’t use --as-needed, so, start using it now.

Similarly to what happened with GCC 4.5 and the combined libpng/berkdb testing I had to clear up the whole tinderbox, and start from scratch after rebuilding the system. But unlike those cases, I didn’t have to rebuild everything in the base system of the tinderbox (in the case of GCC it is needed; in the case of the combined update it was simply faster), so I had the time to look a bit more thoroughly to what happened.

Do you remember my woes with SELinux and libxcrypt? I had a similar surprise, but at least this time it didn’t kick me out of the tinderbox. Again, automagic dependencies, and not just in the base system of the tinderbox but also for packages directly into the system set. And a bunch that are present indirectly.

But first, what on Gentoo is an automagic dependency? Well, an automagic dependency can be either hard or soft… a hard-automagic dependency is a dependency that the upstream build system detect the present of automatically and provides no way to override it; these are the worst situations because it means that you have to hack at the upstream build system to provide an override; it’s for this kind of situations that I started the Automagic fixing guide and it’s one of the first thing that I documented in Autotools Mythbuster so that it can be fixed as intended.

A soft-automagic dependency is one that behaves in the same way but for which upstream actually provided a switch, but the ebuild does not grok and does not export as USE flag properly. This happens because either the ebuild simply lacks the code out of mistake or because the author of the ebuild thought something along the lines of “oh we don’t have foo in Portage, so I cannot set an USE flag, I’ll just leave it as it is”. It is a mistake, a big one, always disable optional features depending on software we don’t have in Portage so that if that software is ever introduced you don’t get yourself an automagic dependency. I’ll show you in a moment that this is exactly what happened.

First problem first, both Python version decided to fail the rebuild with OpenSSL 1 while I did know they should have worked, a quick glance noted that the problem was indirect: dev-lang/tk has an automagic dependency over libXss (the implementation of the XScreenSaver protocol); this obviously made it fail to build the tkinker extension that is needed by so many packages. D’oh! There is also another automagic dependency over a different library, directly in Python, but I hadn’t tracked it down further yet.

I was also able to identify an automagic dependency in qt-sql and another one (unrelated to this cleanup) in qemu; both have been reported together with the ones above.

But what concerned me was an automagic dependency in GNU gettext. This is nasty because this package is part of the system set, and since the system set is what the stages are comprised of, this makes it trouble. While in most cases automagic dependencies are mildly annoying, an exception being the Linux-PAM problem I noted above, and another is system set packages, for the way they are built, by using ROOT= make them vulnerable to automagic dependencies on the host system.

In the case of gettext, the problem was the recently-introduced GNU libunistring, needed for guile and used by gettext and later, if present. It also became quite a trick to properly disable its usage: the macros used to check it out and enable it are taken by that monstrous mess that is called gnulib, and they don’t respond correctly to --without-libunistring-prefix as ./configure would have suggested. On the other hand, --with-included-libunistring worked as a charm, without causing bundled libraries problems because there is no bundled libunistring. And now try telling me again that GNU developers know how to use autotools.

For now, the most important one is covered, without revbump if you’re a guile user and have libunistring installed, you might want to rebuild gettext after a sync, just to be on the safe side.

But learn the lesson, and always --disable and --without the optional dependencies that your package would use if they were present. Do it for you, me and the rest of the users. Pretty please, with sugar!

Comments 8
  1. Are there still downsides to using --as-needed? IIRC, when you wrote about using it you warned that some packages would not work. I’d be happy to file bugs for those packages that I find issues with, but I don’t want to risk messing up my system or the programs I use on a regular basis in the meantime.If I enabled --as-needed and something fails, will it be a build-time failure? If so, will the old version still work with the rest of the system rebuilt?

  2. @IQgryn: you should look up more recent material.. nowdays it’s _very_ much safe; most of the failures remaining are build-time; while we cannot exclude some runtime failure, they are quite rare.

  3. Just as well I’m doing that autotools stuff for Leptonica, lots of hard-automagics in there!

  4. “this is another tricky update if you don’t use –as-needed, so, start using it now.”Are you directing this comment at developers or users? As a user, whenever I add LDFLAGS=”-W1 –as-needed” to /etc/make.conf and try rebuilding my system I get numerous packages failing at the configure stage, usually saying “C compiler can not create executables”.Is there some other way to enable –as-needed that will avoid this problem, or should I file a bug for each affected package?

  5. It’s a lower-case L not a 1: @-Wl,–as-needed@ in @LDFLAGS@. Copy and paste…

  6. The comment above was a typo. My actual make.conf has Wl, not W1.Upon further investigation it seems that have I several packages on my system that will fail if LDFLAGS is set at all, no matter what its value.

  7. You have to make sure you put a comma between entries.LDFLAGS=”-Wl,-O1,–as-needed”I just set it (without comma) and nothing would build. 🙂 I also had better luck with –as-needed last in the list.Running “emerge -ev –keep-going system” right at the moment and everything has built so far.

  8. I don’t know what’s up with my system but setting LDFLAGS *at all*, even LDFLAGS=”” causes a substantial fraction of packages not to build.The specfile technique mentioned here:… works perfectly so far.

Leave a Reply to FlameeyesCancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.