The extraordinary world of libtool

A little sidenote first; since I’ve had more system freezes with Yamato, I’m currently using the Dell laptop as my main box; I’m considering creating a proper frontend for Yamato, and leave that one headless, but that’s longer term, for now I’ll be fine this way. Huge thanks to Jürgen and Cyprien for making these stressful days much more bearable.

In my current line of work, I’ve hit the worst roadblock with, well, PAM. Not on the interface level this time but rather on the build system level. The problem? Building a copy of Linux-PAM 1.1.3 (sys-libs/pam in Gentoo) for a different ROOT will fail, if in that ROOT is installed an older copy of Linux-PAM, and LDFLAGS contains the directive -L${ROOT}/lib (let’s assume 32-bit userland, never 64-bit).

Let’s analyse the current series of problems then: why would you have to mangle LDFLAGS at all? Well, it mostly relates to finding the correct library when linking up. When building within a ROOT directory for the same architecture, most libraries are linked at from the base system, this is why emerge will always install the packages for / as well as for the designated root. When cross-building, the libraries will be searched for in the SYSROOT directory (which usually is /usr/$CHOST), but emerge is not currently smart enough to deal with that case, and thus the packages will not be merged there before merging the one for ROOT. For a previous customer of mine I simply had my wrapper scripts build everything twice, once in SYSROOT and one in the actual filesystem tree.

To avoid the double/triple build of the same package, the common method is to add to the include and library search paths the paths within ROOT that should be searched. This works most of the time, but there are situations where it simply doesn’t work as intended. For instance, doing so with iproute2 will cause -L$ROOT/lib to take precedence over the -L../lib that is used by the build system to link to their internal “libutil” — in turn this will cause the linker to prefer resolving -lutil to the system copy of libutil (provided by glibc) rather than the internal convenience static library. Another good reason to avoid too-common names for convenience libraries. Fixing that was trivial (even though I don’t see it merged yet).

On the other hand, if you were to drop all the mangling of LDFLAGS, you’d be hitting issues with the .la files as libtool will then translate name references into full-paths, referencing / directly, and failing when cross-compiling arm-on-x86. Which is yet another reason why, well, we should be killing those stupid files.

But how does all this relate to Linux-PAM? Well, first of all, just like iproute2 above, the build system for Linux-PAM 1.1.3 uses the combination of library search path and name reference to link to the just-built libpam and libpam_misc, and once again this is trivial to fix (even though it requires a 30KB patch, and breaks my clean record it’s mostly an automated process; I didn’t want to use sed in the ebuild though because I would risk to keep it on new versions where hopefully it won’t be needed). Unfortunately this only fixes half the problem: Linux-PAM builds fine, but it then refuses to install.

The problem here is a pretty common one, which I have already experienced when working on xine and relates to the relinking feature; you probably have seen libtool doing relink during the install phase of a package, often taking quite a lot of time, for instance that’s the case for evolution. The reason why libtool proceeds to take care of this is that you’re building libraries (or in this case, modules/plugins/how-you-want-to-call-them) that link against a just-built library; since libtool can use rpath to make it easier to debug executables from the build directory, it has to make sure that the installed copy is not using broken paths.

I’m not sure if this is a bug or a feature, though, and libtool forces relinking even when the package does not use rpath, or when they are not used at all during build debug (because fast-install is enabled). Also, for just about the same problem with paths, libtool does not use path-based linking during the relink, but rather it uses name references, that are susceptible to the LDFLAGS mangling stated above.

For now, the solution I found is a hack: I simply vouch to libtool that I don’t want it to re-link all the PAM modules during install, which both solves the build problem and should reduce the time needed for Linux-PAM to build. Unfortunately this does not solve it for all the other packages out there with the same problem. I guess it’s time to look for a solution libtool side.

7 thoughts on “The extraordinary world of libtool

  1. I remember investigating this awhile ago, and it seemed that libtool had code to handle this, but for some reason, on a gentoo system. it disabled it.

    Like

  2. > libtool forces relinking even when […] fast-install is enabledThis must be a bug. How can install be fast, if it does not remove the relinking step?(The rationale behind –disable-fast-install is that if you are doing repeated “make && make check” you want to link once per iteration, not twice, so you delay the second link to the time you do “make install”. Instead –enable-fast-install optimizes the case when you run “make && make install” with no intervening “make check”, because testing the program in the build directory would trigger anyway a relink).

    Like

  3. Yes I know what @–enable-fast-install@ was design to solve, but it seems like it doesn’t matter whether I run @make check@ or not, relinking at install time still happens.By the way, does it really need a relink during testing? I don’t think so, given the wrappers wrap around @LD_LIBRARY_PATH@, so it should be able _not_ to relink in that case either. The relink should happen only if the build/testing binaries used rpath, as then you want to drop the rpath during install, not the other way around.

    Like

  4. On my laptop I ended up disabling .la files for everything but packages really needing it (currently, in all the installed packages, only imagemagick really needs it).I once cross-compiled with gentoo, I did all with a user-defined sysroot (not /usr/$CHOST), using the standard prefixes (/,/usr), with a recent version of pkg-config with –sysroot. As I’m doing now with my laptop, at that time I removed all .la files and had no problem with libtool, it even compiled Xorg.As a sidenote, I don’t understand why you are talking about cross-compilation and installing both in SYSROOT and ROOT? If it’s cross-compilation you obviously can’t install the packages to /

    Like

  5. When doing cross-compile with Gentoo you set the ROOT variable where the newly created filesystem will be. So for instance in the case of ChromiumOS you’ll have ROOT=/build/arm-generic for building the ChromiumOS filesystem for ARM.

    Like

  6. Did you ever figure out anything more about this? For example installation of gegl also takes _ages_ because it relinks every operation .so which is over 200 files. I looked at a generated configure and it has this code; # This must be glibc/ELF. linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) # boring stuff # … # This implies no fast_install, which is unacceptable. # Some rework will be needed to allow for fast_install # before this can be enabled. hardcode_into_libs=yesSo it appears fast install is always disabled on current libtools? I would also imagine this is becoming more problemating with -flto since this linking stage is in fact the entire compilation now.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s