Autotools Mythbuster: so why do we have three projects?

As much as I’ve become an expert on the topic, there is one question I still have no idea how to answer, and that is why on earth we have three separate projects (autoconf, automake, libtool) instead of a single Autotools project. Things get even more interesting when you think that there is the Autoconf Archive – which, by the way, references Autotools Mythbuster as best practices – and then projects such as dolt that are developed by completely separate organisations.

I do think that this is a quite big drawback of autotools compared to things like CMake: you now have to allow for combinations of different tools written in different languages (autoconf is almost entirely shell and M4, automake uses lots of Perl, libtool is shell as well), with their own deprecation timelines, and with different distributions providing different sets of them.

My guess is that many problems lie in the different sets of developers for each project. I know for instance that Stefano at least was planning to have a separate Automake-NG implementation that did not rely on Perl at all, but used GNU make features, including make macros. I generally like this idea, because similarly to dolt it removes overhead for the most common case (any Linux distribution will use GNU make by default), while not removing the option where this is indeed needed (any BSD system.) On the other hand it adds one more dimension to the already multi-dimensional compatibility problem.

Having a single “autotools” package, while making things a bit more complicated on the organizational level, could make a few things fit better. For instance if you accepted Perl as a dependency of the package – since automake needs it; but remember this is not a dependency for the projects using autotools! – you could simplify the libtoolize script which is currently written in shell.

And it would probably be interesting if you could just declare in your configure.ac file whether you want a fully portable build system, or you’re okay with telling people that they need a more modern system, and drop some of the checks/compatibility quirks straight at make dist time. I’m sure that util-linux does not care about building dynamic libraries on Windows, and that PulseAudio does not really care for building on non-GNU make implementations.

Of course these musings are only personal and there is nothing that substantiate them regarding how things would turn out; I have not done any experiment with actually merging the packages into a single releasable unit, but I do have some experience with split-but-not-really software, and in this case I can’t see many advantages in the split of autotools, at least from the point of view of the average project that is using the full set of them. There are certainly reasons for which people would prefer them to be split, because especially if they have been using only autoconf and snobbing automake all this time, but… I’m not sure I agree with those reasons to begin with.

Autotools Mythbuster: who’s afraid of libtool?

This is a follow-up on my last post for autotools introduction. I’m trying to keep these posts bite sized both because it seems to work nicely, and because this way I can avoid leaving the posts rotting in the drafts set.

So after creating a simple autotools build system in the previous now you might want to know how to build a library — this is where the first part of complexity kicks in. The complexity is not, though, into using libtool, but into making a proper library. So the question is “do you really want to use libtool?”

Let’s start from a fundamental rule: if you’re not going to install a library, you don’t want to use libtool. Some projects that only ever deal with programs still use libtool because that way they can rely on .la files for static linking. My suggestion is (very simply) not to rely on them as much as you can. Doing it this way means that you no longer have to care about using libtool for non-library-providing projects.

But in the case you are building said library, using libtool is important. Even if the library is internal only, trying to build it without libtool is just going to be a big headache for the packager that looks into your project (trust me I’ve seen said projects). Before entering the details on how you use libtool, though, let’s look into something else: what you need to make sure you think about, in your library.

First of all, make sure to have an unique prefix to your public symbols, be them constants, variables or functions. You might also want to have one for symbols that you use within your library on different translation units — my suggestion in this example is going to be that symbols starting with foo_ are public, while symbols starting with foo__ are private to the library. You’ll soon see why this is important.

Reducing the amount of symbols that you expose is not only a good performance consideration, but it also means that you avoid the off-chance to have symbol collisions which is a big problem to debug. So do pay attention.

There is another thing that you should consider when building a shared library and that’s the way the library’s ABI is versioned but it’s a topic that, in and by itself, takes more time to discuss than I want to spend in this post. I’ll leave that up to my full guide.

Once you got these details sorted out, you should start by slightly change the configure.ac file from the previous post so that it initializes libtool as well:

AC_INIT([myproject], [123], [flameeyes@flameeyes.eu], [https://flameeyes.blog/tag/autotools-mythbuster/])
AM_INIT_AUTOMAKE([foreign no-dist-gz dist-xz])
LT_INIT

AC_PROG_CC

AC_OUTPUT([Makefile])

Now it is possible to provide a few options to LT_INIT for instance to disable by default the generation of static archives. My personal recommendation is not to touch those options in most cases. Packagers will disable static linking when it makes sense, and if the user does not know much about static and dynamic linking, they are better off getting everything by default on a manual install.

On the Makefile.am side, the changes are very simple. Libraries built with libtool have a different class than programs and static archives, so you declare them as lib_LTLIBRARIES with a .la extension (at build time this is unavoidable). The only real difference between _LTLIBRARIES and _PROGRAMS is that the former gets its additional links from _LIBADD rather than _LDADD like the latter.

bin_PROGRAMS = fooutil1 fooutil2 fooutil3
lib_LTLIBRARIES = libfoo.la

libfoo_la_SOURCES = lib/foo1.c lib/foo2.c lib/foo3.c
libfoo_la_LIBADD = -lz
libfoo_la_LDFLAGS = -export-symbols-regex '^foo_[^_]'

fooutil1_LDADD = libfoo.la
fooutil2_LDADD = libfoo.la
fooutil3_LDADD = libfoo.la -ldl

pkginclude_HEADERS = lib/foo1.h lib/foo2.h lib/foo3.h

The _HEADERS variable is used to define which header files to install and where. In this case, it goes into ${prefix}/include/${PACKAGE}, as I declared it a pkginclude install.

The use of -export-symbols-regex ­– further documented in the guide – ensures that only the symbols that we want to have publicly available are exported and does so in an easy way.

This is about it for now — one thing that I haven’t added in the previous post, but which I’ll expand in the next iteration or the one after, is that the only command you need to regenerate autotools is autoreconf -fis and that still applies after introducing libtool support.

Dealing with insecure runpaths

Somehow, lately I’ve had a few more inquiries about runpaths than usual — all for different packages, which makes it quite a bit more interesting. Even though not all the questions regarded Gentoo’s “insecure runpaths” handling, I think it might be well worth it to write a bit more about it. Before going into details about it, though, I’ll point you to my previous post for a description of what runpaths (and rpath) are, instead of repeating it all here.

So, what is an insecure runpath? Basically any runpath set to a directory where an attacker may have control is insecure, because it allows an attacker to load an arbitrary shared object which can then run arbitrary code. The definition of attacker, as I already discussed is flexible depending on the situation.

Since runpaths are, as the name suggest, paths on the filesystem, there are two main starting points that would cause a path to become insecure: runpaths derived from the current working directory, and runpaths derived from any world-writable directory. The ability for an attacker to place the correct object in the correct path varies considerably, but it’s a good rule of thumb to consider the both of them just as bad. What happens for packages built in Gentoo often enough, is for the runpath to include the build directory in its runpath, which could be either /var/tmp or /tmp (if you, for instance, build in tmpfs — please tell me you’re not using /dev/shm for that!).

Depending on how your system is set up, this might not be as insecure as it might sound at first, because for instance in my laptop, /var/tmp/portage is always present, and always owned by the portage user, which makes it less vulnerable to a random user wanting to move there a particularly nasty shared object… on the other hand, assuming that it’s secure enough is the wrong move, full stop.

Before I start panicking people — Portage is smarter than you think and it has been, by default, stripping insecure runpaths for a while. Which means that you don’t have to fret about fixing the issues when you see the warning — but you really should look into fixing them for good. I would also argue that since Portage already strips the runpaths at install time, you shouldn’t just use chrpath to remove the insecure paths after the build, but you should either fix it properly or leave it be, in my opinion. This opinion might be a bit too personal, as I don’t know if pkgcore or paludis support the same kind of fixing.

So let’s go back to the two sources of insecurity in the runpaths. In the case of a current directory being the base for the insecure path, which means having an rpath of either . or ../something, this is usually bad logic in the way the package tries to set its rpath, as instead of the current work directory, the developers most likely wanted to refer to where the executable itself is installed, which is, instead, $ORIGIN literally. Another common situation is for that literal to be treated as a variable name, so that $ORIGIN/../mylibs becomes something like /../mylibs which is also wrong.

But a much more common situation arises when our build directory is injected into the runpath. Something along the lines of /var/tmp/portage/pkg-category/package-0.0.0/image/usr/lib64/mypkgslibs — more rarely it would point to the build directory rather than the image directory. In many cases this happens because the upstream build system does not know about, or mishandles, the DESTDIR variable.

The DESTDIR variable is commonly used to install a software package at a given offset — binary distributions will then generate a binary package out of it, source distributions like Gentoo will then merge the installed copy to the live filesystem after recording which files have been installed. In either case, the understanding behind this variable is that the final location of the executables will not include it. Unfortunately not all build systems do support it, so in some cases we end up doing something a bit more hackish by replacing /usr with ${D}/usr in the definition of the install prefix. The prefix is, though, commonly used to identify where the executables will be at the end, so it would be possible for a build system to have, in the parameters, -rpath ${prefix}/lib/mylibs which would then inject ${D} on the runpath.

As you can see, for most common situations it’s a matter of getting upstream to fix their build system. In other cases, the problem is that the ebuild is installing files without going through the build system’s install phase, which, at least with libtool, would often re-link the object files to make sure the rpath is handled correctly.

Beside this, there isn’t much more I can add, I’m afraid.

Library SONAME bumps and .la files: some visual clues

Before going on with the post, I’ll give users who’re confused by the post’s title some pointers on how to decipher it: I discussed .la files extensively before, and you can find a description of SONAMEs in another post of mine.

Long- and medium-time Gentoo users most likely remember what happened last time libpng was bumped last year, and will probably worry now that I’m telling them that libpng 1.5 is almost ready to be unmasked (I’m building the reverse dependencies in the tinderbox as we speak to see what breaks). Since I’ve seen through it with the tinderbox, I’m already going to tell you that it’s going to hurt, as a revdep-rebuild call will ask you to rebuild oh-so-many packages due to .la files that, myself, I’ll probably take the chance to move to the hardened compiler and run an emerge -e world just for the kicks.

But why is it this bad? Well, mostly it is the “viral propagation” of dependencies in .la files, which by itself is the reason why .la files are so bad. Since libgtk links to libcairo, and libcairo to libpng, any other library linking with libgtk will be provided with a -lpng entry to link to libpng, no matter whether it uses it or not. Unfortunately, --as-needed does not apply to libtool archives, so they end up overlinking, and only the link editor can drop the unused libraries.

For the sake of example, Evolution does not use libpng directly (the graphic files are managed through GTK’s pixbuf interface), but all of its plugins’ .la files will refer to libpng, which in turn means that revdep-rebuild will pick it up to rebuild it. D’oh!

So what about the visual clue? Well, I’ve decided to use the data from the gold based tinderbox to provide a graph of how many ELF objects actually link to the most common libraries, and how many libtool archives reference them. The data wasn’t easy to extract, mostly because at a first glance, the .la files seemed to be dwarfed by the actually linked objects.. until I remembered that ELF executable can’t have a corresponding .la file.

Library linking histogram

I’m sorry of some browsers might fail to display the image properly; please upgrade to a decent, modern browser as it’s a simple SVG file. The gnuplot script and the raw data file are also available if you wish to look at them.

The graph corroborates what I’ve been saying before, that the bump of libraries such as libexpat and libpng only is a problem because of overlinking and .la files. Indeed you can see that there are about 500 .la files listing either of the two libraries, when there are fewer than a hundred shared objects referencing them. And for zlib it’s even worse: while there are definitely more shared objects using it (348), there are four times as many .la files listing it as one of the dependencies, for no good reason at all.

A different story applies to GLib and GTK+ themselves: the number of shared objects using them is higher than the number of .la files that list them among their dependencies. I guess the reason here is that a number of their users are built with non-libtool-based build systems, and another good amount of .la files are removed by the less lazy Gentoo packagers (XFCE should be entirely .la free nowadays, and yes, it links to GTK+).

Now it is true that the amount of .la files and ELF files is not proportional to the number of packages installing them (for instance Evolution installs 24 .la files and 69 ELF objects), so you can’t really say much about the number of packages you’d have to rebuild when one of the three “virulent” libraries (libpng, libexpat, libz) is installed, but it should still be clear that marking five hundreds files as broken simply because they list a library that is gone, without their respective binary actually having anything to do with said library, is not the best approach we can have.

Dropping the .la file for libcairo (which is where libgtk picks it up) should probably make it much more resilient to the libpng bumps, which have proven to be the nastiest ones. I hope somebody will step up to do so, sooner or later.

Gold readiness obstacle #5: libtool (part 2, OpenMP)

After some digression on the issues with the latest glibc version, it’s time for the fifth instalment of gold readiness for the Gentoo tree, which is completing the libtool issues that I noted a couple of days ago in part 1.

As I said, this relates to a known issue with libtool and OpenMP, to the point that there is a Gentoo bug open and upstream libtool package is already fixed to deal with this, it’s just not trickled down to a release and from there into Gentoo. Although I guess it might be a good idea to just apply this with elibtoolize as I’ve done for the configure fix for gold.

What is the problem? Well, the first problem is with the design of OpenMP language extensions, and with some other flags that implicitly enable those extensions. While these are flags that relate to the GCC frontend (gcc command), they not only change the semantics of the compiled code, they also change the semantics of the linking step, by adding a link to the OpenMP implementation library. This means that the frontend needs to know about this both at compile time as well as link time (where the frontend converts it to the proper linking flags for the link editor to pick up OpenMP).

Unfortunately, when using libtool, its script is parsing and mangling the options first (it’s for this reason that libtool had to be patched to get --as-needed working as intended). Up to now, it didn’t know that -fopenmp should have been passed to the linking frontend of gcc just the same.

Okay, in truth this is not much of an issue for gold only; it is just the same issue when using ld/bfd. But the switch to a link editor that has stricter underlinking rejection makes the issue much more apparent, in particular because, while libtool is usually involved in building the libraries, there is no reason (beside the slight chance of using static linking through libtool archives) to force its usage for building final executables, which means that a single -fopenmp at the final linking point would be quite enough.

Gold readiness obstacle #4: libtool (part 1)

In my current series of posts about gold this time I’m presenting you with a two-parter that shows how GNU libtool is causing further problem with this new link editor. The reason why I split this into two part is because it hits two different issues with it: one is a “minor inconvenience” due to its design, and the other is a known bug due to it.

You might remember me distinguishing into two schools of build configuration systems one relying on tests being compiled and executed, and the other relying on knowing intimate details about the working of the tools to be used with it. Autotools for the vast part are designed to fall squarely into the former category, with a number of advantages, but also of disadvantages, the most obvious of which is the slowness of the ./configure process.

This is true of moth autoconf and autoconf macro; on the other hand, libtool works vastly as a repository of knowledge, knowing rules about various operating systems, link editors and compilers. It shouldn’t surprise anybody, given that it already spends way more time than one would like, to discover how to build shared libraries; if it was a pure guessing game, it would probably make it unbearable to use.

Unfortunately this makes libtool vulnerable to the main issue of knowledge repository systems, with a bad twist: not only it becomes outdated as operating systems, link editors and compilers are released, but due to the autotools philosophy of not requiring the tools themselves at build time, simply updating the system copy of libtool is not enough. For details about this statement see my previous post on libtool from which I still haven’t had time to distil documentation for my guide.

What this boils down to is simple yet scary: even though libtool properly implemented support for gold starting from version 2.2.7 – which actually only means that it now knows that gold supports anonymous versions in linker scripts – any package whose autotools had been built with older versions wouldn’t support gold out of the box. Luckily for us, this isn’t an excessively invasive issue: anonymous versioning, as far as I can tell, is only used when using libtool to export and hide symbols which for good or bad is not used that often.

To make sure that projects know about gold support for the feature, you’re left with two solutions. The former is the obvious one of rebuilding autotools; while this is often necessary for other reasons, it isn’t that good an idea, because it wastes time. I have other notes about the rebuilding of autotools but I’ll skip over them for now. In Gentoo we have already a method to take care of this, consisting of the elibtoolize function. This function applies a number of patches over an already-generated autotools build system, to fix common issues (mostly due to libtool) without having to rebuild autotools altogether.

My first encounter with this interface was due to my early Gentoo/FreeBSD work. One of the things libtool “knows” about, is that FreeBSD does not like having two-part sonames which made many minor version bump, which should have kept the same, or a compatible, ABI into a link-breaking update. Since we didn’t look for binary compatibility with the original FreeBSD I went out of my way to patch libtool so that it would use the Linux naming scheme instead. Given that I already knew how to set it up, it wasn’t that difficult to add one extra patch when gold is used as the link editor of choice.

Unfortunately, having worked with that before also means I know the bad side of elibtoolize: too many packages do not use it at all. This is partly because developers do not know about the need for it (it applies, among others, the --as-needed compatibility patch), and partly because they think that it causes autotools to be rebuilt, rather than preventing the requirement for it.

Waiting for other obstacles to be solved, this one is probably going to be bothering us for months, if not years, to come. I don’t expect a resolution anytime soon, given that the idea of having a comprehensive, versioned system for autopatching packages, which was requested/proposed many times, never went into fruition. Maybe I’ll try pushing for it for the next GSoC (I note that I only partecipate into even years editions).

Parameters to ./configure can deceive

As much as I’d like for this not to be the case, one of the most obnoxious problems with autotools is that not only there is little consistency between packages on the use of ./configure parameters, but also when there is, the parameters themselves can deceive, either because of their sheer name or because of one package differing in its use.

This is the reason why I dislike the idea of autotools-utils.eclass of wiring the debug USE flag to --enable-debug: that option is often implemented badly, if at all, and is not consistently used across projects: it might enable debug specific code, it might be used to turn off assertions (even though AC_HEADER_ASSERT already provides a --disable-assert option), it might add debug symbol information (-g and -ggdb) or, very bad!, it might fiddle with optimizations.

One of the most-commonly deceiving options is --enable-static, which is provided by libtool, and thus used in almost any autotools-based build system. What this does is tell libtool to build static archives (static libraries) for the declared libtool targets, but enough people, me included when I started, assumed it was going to tell the build system to build static binaries, and wired it to the static USE flag, which was simply wrong. Indeed, most of the static USE flags are wired to append -static to LDFLAGS instead, while --enable-static is tied to static-libs, when it does make sense — in a lot of cases, building the static libraries doesn’t make sense; for details see my other post where I make some notes about internal convenience libraries.

Unfortunately, while libtool-based packages are forced into consistency on this parameter, this doesn’t stop it from being inconsistent among other, non-libtool-based projects (who intend it to behave as “make the final executable static”) and, ironically, by libtool itself. Indeed if you use the --disable-static option to ./configure for libtool itself, what you’re actually asking is for the libtool script to be unable to build static libraries altogether. This is not happening out of the blue though; in itself it is actually fairly consistent within libtool itself. But to understand this you need to take a step back.

Autoconf, automake, and libtool are all designed not to add extra dependencies to build a package, unless you change the autotools sources themselves. The fact that for a distribution such as Gentoo this is so common that building a system without autotools is near impossible is a different story altogether. Following this design, when a package uses autotools and libtool, it cannot rely on the presence of /usr/bin/libtool; what it does instead is creating its own libtool script, using the same macros used by the libtool package. Incidentally, this is the reason why we can’t just patch the libtool package to implement features and work around bugs, but we usually have to add a call to elibtoolize in the ebuild.

So, from a behaviour point of view, the --disable-static option does nothing more than generating a script that is unable to build static libraries, and from there you get packages without static libraries, when they don’t use the system libtool script.

On the other hand, the system libtool script is still used by some packages, one of which turns out to be lua. The build system used by lua is one that is quite messed up, and it relies on the creation of a static liblua archive, from which just a subset of sources are used to generate the lua compiler. To do so, it uses the system libtool script. If your sys-devel/libtool package is built with --disable-static, though, you get failures which took me a while to debug, when it hit me on the chroot I use to build package for my vservers (it is related to ModSecurity 2.6 having an automagic dependency on lua I haven’t had time to work on yet).

What’s the end of the story for this post? Do not assume that two packages share the same meaning for a given option unless those packages come from the same developer, I guess. And even then make sure to check it out. And don’t provide random parameters to packages in eclasses or generic setting files just because it seems like a good idea, you might end up debugging some random failure.

P.S.: I’ll try to find time to write something about libtool’s system script issue on Autotools Mythbuster over the weekend; last week I added some notes about pkg-config and cross-compiling due to work I’ve been doing for one of my contract jobs.

Surviving without libtool archives

The sheer amount of articles regarding libtool archives in my blog’s archive is probably enough to tell that I have a pretty bad relationship with them. But even though I have been quiet, lately, I haven’t stopped working on them.

In particular, my current “daily job”, or at least one of them, has among its task the one of getting more reliable cross-compilation support with Gentoo’s ebuilds, and dropping the cursed .la files is part of the plan. With them around, often enough, -L/usr/lib64 is added to the linking line, which causes all sorts of troubles once it’s resolved to a fully qualified path such as /usr/lib64/libfoo.so which cannot obviously be linked against.

After a bit of doubts on how to proceed (waiting for Gentoo proper to remove all of them, given the kind of obstruction I find coming from developers, was not really an option), it was decided to try building the whole chroot used for cross-compilation, and the cross-compiled roots by removing all the .la files with the exclusion of the one that is known to be required (libltdl.la).

For those wondering, it was achieved with this “simple” setting in make.conf:

INSTALL_MASK="
/usr/lib/lib[0-9]*.la
/usr/lib/lib[a-k]*.la
/usr/lib/lib[m-z]*.la
/usr/lib/libl[0-9]*.la
/usr/lib/libl[a-s]*.la
/usr/lib/libl[u-z]*.la
/usr/lib/liblt[0-9]*.la
/usr/lib/liblt[a-c]*.la
/usr/lib/liblt[e-z]*.la
/usr/lib/libltd[0-9]*.la
/usr/lib/libltd[a-k]*.la
/usr/lib/libltd[m-z]*.la
/usr/lib/libltdl[0-9]*.la
/usr/lib/libltdl[a-z]*.la
"

At this point, my opinion has always been that there would be no problem with building further packages even though the libtool archives were gone… turns out I was mostly, but not entirely, right. Indeed if the system were built using the standard GNU ld linker, there would have been no package failure; on the other hand, all of this is built using gold ­– which is much stricter about underlinking – and that makes a huge difference.

Not only the use of libtool by itself would make it mostly pointless to use --as-needed, but it also makes --no-add-needed (the feature that makes the linker strict in terms of underlinking) much less effective: if you only link to libfoo.la, which in turn lists -lbar, and you use symbols from both, the libtool archive would provide both to the linker, hiding the fact that you didn’t express your dependencies properly.

But out of a whole operating system built without .la files, how many packages did require fixes? The answer is two: libsoup (which was actually already fixed in the second-to-last release, so the fix was simply updating the version used) and tpm-tools (that, similarly to trousers and opencryptoki has a quite bogus build system).

I’m not saying that they would be the only packages suffering from these issues, and in particular, with the fact that this system is not building anything statically, it is likely to encounter much fewer complications, but it is more than likely that with minimal effort we’d suffer fewer problems with linking, rebuilds, and dependencies if we were to drop those files entirely, and switch to fixing the few packages failing.

The extraordinary world of libtool

A little sidenote first; since I’ve had more system freezes with Yamato, I’m currently using the Dell laptop as my main box; I’m considering creating a proper frontend for Yamato, and leave that one headless, but that’s longer term, for now I’ll be fine this way. Huge thanks to Jürgen and Cyprien for making these stressful days much more bearable.

In my current line of work, I’ve hit the worst roadblock with, well, PAM. Not on the interface level this time but rather on the build system level. The problem? Building a copy of Linux-PAM 1.1.3 (sys-libs/pam in Gentoo) for a different ROOT will fail, if in that ROOT is installed an older copy of Linux-PAM, and LDFLAGS contains the directive -L${ROOT}/lib (let’s assume 32-bit userland, never 64-bit).

Let’s analyse the current series of problems then: why would you have to mangle LDFLAGS at all? Well, it mostly relates to finding the correct library when linking up. When building within a ROOT directory for the same architecture, most libraries are linked at from the base system, this is why emerge will always install the packages for / as well as for the designated root. When cross-building, the libraries will be searched for in the SYSROOT directory (which usually is /usr/$CHOST), but emerge is not currently smart enough to deal with that case, and thus the packages will not be merged there before merging the one for ROOT. For a previous customer of mine I simply had my wrapper scripts build everything twice, once in SYSROOT and one in the actual filesystem tree.

To avoid the double/triple build of the same package, the common method is to add to the include and library search paths the paths within ROOT that should be searched. This works most of the time, but there are situations where it simply doesn’t work as intended. For instance, doing so with iproute2 will cause -L$ROOT/lib to take precedence over the -L../lib that is used by the build system to link to their internal “libutil” — in turn this will cause the linker to prefer resolving -lutil to the system copy of libutil (provided by glibc) rather than the internal convenience static library. Another good reason to avoid too-common names for convenience libraries. Fixing that was trivial (even though I don’t see it merged yet).

On the other hand, if you were to drop all the mangling of LDFLAGS, you’d be hitting issues with the .la files as libtool will then translate name references into full-paths, referencing / directly, and failing when cross-compiling arm-on-x86. Which is yet another reason why, well, we should be killing those stupid files.

But how does all this relate to Linux-PAM? Well, first of all, just like iproute2 above, the build system for Linux-PAM 1.1.3 uses the combination of library search path and name reference to link to the just-built libpam and libpam_misc, and once again this is trivial to fix (even though it requires a 30KB patch, and breaks my clean record it’s mostly an automated process; I didn’t want to use sed in the ebuild though because I would risk to keep it on new versions where hopefully it won’t be needed). Unfortunately this only fixes half the problem: Linux-PAM builds fine, but it then refuses to install.

The problem here is a pretty common one, which I have already experienced when working on xine and relates to the relinking feature; you probably have seen libtool doing relink during the install phase of a package, often taking quite a lot of time, for instance that’s the case for evolution. The reason why libtool proceeds to take care of this is that you’re building libraries (or in this case, modules/plugins/how-you-want-to-call-them) that link against a just-built library; since libtool can use rpath to make it easier to debug executables from the build directory, it has to make sure that the installed copy is not using broken paths.

I’m not sure if this is a bug or a feature, though, and libtool forces relinking even when the package does not use rpath, or when they are not used at all during build debug (because fast-install is enabled). Also, for just about the same problem with paths, libtool does not use path-based linking during the relink, but rather it uses name references, that are susceptible to the LDFLAGS mangling stated above.

For now, the solution I found is a hack: I simply vouch to libtool that I don’t want it to re-link all the PAM modules during install, which both solves the build problem and should reduce the time needed for Linux-PAM to build. Unfortunately this does not solve it for all the other packages out there with the same problem. I guess it’s time to look for a solution libtool side.

The automake and libtool clash

I’ve been warned today by my friend lavish that there are packages, such as ettercap and tvtime that are failing because of a clash between libtool and older automake versions. Indeed, that is the case and I’m now on the case to resolve it.

The problem here is that libtool 2.4 requires that you use it with at least automake 1.9, so anything using 1.7 (ettercap) or 1.8 (tvtime) will fail with a number of aclocal errors. There are internal changes forcing this, it’s nothing new and nothing extraordinary. Unfortunately, autotools.eclass allows maintainers to require an older automake version. This has been extensively used up to now to avoid future-version compatibility with automake… but it’s now biting our collective asses with this.

The solution, obviously, is to make sure that ebuilds are tested against a more modern version of automake, and that is usually quite quick when using 1.8 or later, older versions might be more of a problem; turns out that I suggested to start working on this back in June 2009 — magic? ESP? Or simple knowing my tools by now?

I’m now going to work through the tree to take care of these issues, since they are quite important, but also relatively easy to fix. Plus they require autotools expertise and I’m positive that at last half the involved maintainers would be asking me to fix them anyway.

Now, the question is why didn’t my tinderbox catch this beforetime? The answer is easy: *it is not running*… between the problems and the gold f-up – and few to nobody interested in helping me – I haven’t ran the tinderbox for the past two weeks.

And yet this is the exact kind of breakage it is supposed to find. So please consider how much work I’ve been pouring in to protect us all from this stuff happening, and don’t blame me if I actually come asking for help from time to time to pay for the hardware — there are requests for specific CPU and memory hardware to upgrade the workstation on the page I listed above, if you feel like contributing directly at running it; I should add soon a request for two Caviar Black drives where most builds happen (the two Samsung SpinPoint I have start to report relocated sectors, they are at the end of their life here), or two RE4 for the operating system/work data (which is now filled to its limits).

Edit: since the system locked-up during the build, likely caused by the HDDs failing, I had to order the new ones… at a moment where I’m €1550 short to pay taxes, I had to spend another €190 for the new disks. Can you tell now why I’d wish that at least the rest of the developers would help me reducing the overhead of tinderbox by at least doing some due diligence in their packages, rather than complain if I ask for help to pay the bills&hardware of the tinderbox? — No, having the Foundation reimburse me of the expenses is not going to help me any further since I’d have to pay extra taxes over that.

Anyway, for now I think I’ll be limiting the tinderbox to two cores out of the eight I have available, and keep running it in the background, hopefully it will not cause the rest of my system (which is needed for my “daily” job — I say “daily” because I actually spend evenings and nights working on it) to hang or perform too badly.

Anyway, 6am, I haven’t slept, and I still have a number of things to do…