This Time Self-Hosted
dark mode light mode Search

Avoid building (some) static libraries

Warning: the following trick is the kind of material that allows you to shoot yourself in the foot. On the other hand, since there isn’t enough push in Gentoo to solve the situation for good, I’ve decided to use this extensively myself.

Servers and embedded have two completely opposite conception of utility for libraries; in the latter case you most likely prefer static archives – when a single package use them at least – while in the former you prefer shared objects, even if it’s a single package, even better if it’s shared by multiple packages. This is mostly because of the difference in update policies: you don’t update an embedded system that is working unless you really have to; you update a server as soon as a security issue is found.

So, if shared objects are preferred for my own server, why should I have static archives laying around? Well I shouldn’t; especially on the server itself. Now, to remove static archives and include files on the server side, when using binary packages to upgrade the server, it’s actually quite easy: you just have to INSTALL_MASK them in the server’s configuration files (but not on the build chroot system you use to generate them). It solves the problem of using up space for no good reason but it still packages them in the the .tbz2 files that are sent to the server.

Now, for include files there is little you can do, since you do need on the build system, and you cannot really just INSTALL_MASK the static archives away entirely, because things like flex or libatomic_ops don’t install anything else but a static archive. What you can do is use the per-package environment support to remove the static archives (and, most of the time, the libtool files that are unneeded):

# /etc/portage/env/dev-libs/libxml2
export INSTALL_MASK="${INSTALL_MASK} /usr/lib*/*.a /usr/lib*/*.la"

This will be parsed at each phase, and will avoid merging in the static and libtool archives in the system and the tarballs. You just have to create a file like this for each of the packages that install unwanted static archives. But this will still compile two copies of every source file at build time, one with PIC and one without, to properly create static and shared objects. While solving this in a totally generic fashion is basically not possible, you can make use of some features of standard autoconf-based packages, starting from EXTRA_ECONF:

# /etc/portage/env/dev-libs/libxml2

# don't build static archive
export EXTRA_ECONF="${EXTRA_ECONF} --disable-static"

# get rid of the (useless) libtool archive
export INSTALL_MASK="${INSTALL_MASK} /usr/lib*/*.la"

This will pass the --disable-static option to econf, like most of the static-libs USE flags do. It only works for standard autoconf packages so it fails for stuff like OpenSSL and ncurses, for which you don’t have a proper escape route. On the other hand, it does its job for the most common packages. This trick gets also pretty useful when you have to deal with EC2, as the double-build is one heck of a killer.

Try not to abuse of EXTRA_ECONF, though. It actually lets you change ebuilds without having to hack eclasses or ebuilds altogether, but as I said, it’s one very good way to shoot yourself in the foot.

Comments 5
  1. I have to flu so maybe my reading comprehension is impaired but my understanding of this:bq. “Servers and embedded have two completely opposite conception of utility for libraries; in the former case you most likely prefer static archives – when a single package use them at least – while in the latter you prefer shared objects, even if it’s a single package, even better if it’s shared by multiple packages.”is that servers (“the former”) prefer static archives and embedded (“the latter”) prefer shared objects. But my reckoning, and the next paragraph, mean the opposite. Switch “former” and “latter”?Also: s/wtih/with/

  2. Thanks Zeev… yeah I totally got the two switched… sigh, I start to feel the migraines lately are incapacitating my concentration skills 🙁

  3. I use a different approach to fix the problem, instead to use Portage’s environment profiles (which is a tedious job if you have several thousands of packages installed), I use a simple shell script that launch emerge with EXTRA_ECONF / INSTALL_MASK variables exported globally, something like this:—–workstation / # cat /usr/bin/femerge #!/bin/sh#disable static archivesEXTRA_ECONF=”${EXTRA_ECONF} –disable-static”#compile only PIC objects, skipping non-PIC objects#EXTRA_ECONF=”${EXTRA_ECONF} –with-pic”[CUT]EXTRA_ECONF=”${EXTRA_ECONF}” INSTALL_MASK=”${INSTALL_MASK}” emerge “$@”—–This solution is good for servers / tinderboxes where the global compilation of static libraries is completely useless and the global removal of libtool archives is harmless. Then I use Portage’s environment profiles just to tweak compilation for that packages that misuse static/lafiles (really few packages).With this trick I have saved over ~600MB (sometime over 800MB) of useless static archives from every Gentoo Boxes and their backups (static archives are useless inside the backups too).p.s.: ffmpeg’s ebuild forces –enable-static, is there a good reason to do it? (just asking for curiosity, nothing else)

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.