Problems and mitigation strategies for –as-needed

Doug and Luca tonight asked me to comment about the --as-needed by default bug. As you can read there, my assessment is that the time is ready to bring on a new stage of --as-needed testing through use of forced --as-needed on specs files. If you wish to help testing with this (and I can tell you I’m working on testing this massively), you can do it by creating your own asneeded specs.

Note, Warning, End of the World Caution!

This is not a procedure for the generic users, this is for power users, who don’t mind screwing up their system even beyond repair! But it would help me (and the rest of Gentoo) testing.

First of all you have to create your own specs file, so issue this command:

# export SPECSFILE=$(dirname "$(gcc -print-libgcc-file-name)")/asneeded.specs
# export CURRPROFILE=/etc/env.d/gcc/$(gcc-config -c)
# gcc -dumpspecs | sed -e '/link:/,+1 s:--eh-frame-hdr: --as-needed:' > "$SPECSFILE"
# sed "${CURRPROFILE}" -e '1iGCC_SPECS='$SPECSFILE > "${CURRPROFILE}-asneeded"
# gcc-config "$(basename "${CURRPROFILE}")-asneeded"
# source /etc/profileCode language: PHP (php)

This will create the specs file we’re going to use, it just adds —as-needed for any linking command that is not static, then it would create a new configuration file for gcc-config pointing to that file. After this, —as-needed will be forced on. Now go rebuild your system and file bugs about the problems.

A package known to break with this is xmlrpc-c, which in turn makes cmake fail; as some people lack common sense (like adding a way to not have xmlrpc-c as a dependency because you might not want your cmake to ever submit the results of tests), this can get nasty for KDE users. But maybe, just maybe, someone can look into fixing the package at this point.

But xmlrpc-c does require some reflection on how to handle --as-needed in these cases: the problem is that the error is in one package (xmlrpc-c) and the failure in another (cmake) which makes it difficult to asses whether --as-needed break something; you might have a broken package, but nothing depending on it, and never notice. And a maintainer might not notice that his package is broken because other maintainers will get the bugs first (until they get redirected properly). Indeed it’s sub-optimal.

Interestingly enough, Mandriva actually started working on trying to resolve this problem radically, they inject -Wl,--no-undefined in their build procedures so that if a library is lacking symbols, the build dies sooner rather than later. This is fine up to a certain point, because there are times when a library does have undefined symbols, for instance if it has a recursive dependency over another library (which is the case of PulseAudio’s libpulse and libpulsecore, which I discussed with Lennart some time ago). Of course you can work this around by adding a further -Wl,--undefined that then tells ld to discard the former, but it requires more work, and more coordination with upstream.

Indeed, coordination with upstream is a crucial point here, since having to maintain --as-needed fixes in Gentoo is going to be cumbersome in the future, and even more if we start to follow Mandriva’s steps (thankfully, Mandriva is submitting the issues upstream so that they get fixed). But I admit I also haven’t been entirely straight on that; I pushed just today a series of patches to ALSA packages, one of which, to alsa-tools, was for --as-needed support (the copy we have in Portage also just works around the bug rather than fixing it). Maybe we need people that starts checking the tree for patches that haven’t been pushed upstream and tries to push them (with proper credits of course).

Another thing that we have to consider is that many times we have upstream that provide broken Makefiles, and while sending the fix upstream is still possible, fixing it in the ebuild takes more time than it is worth; this is why I want to refine and submit for approval my simple build eclass idea, that at least works as a mitigation strategy.