LXC banes — libraries, message queues and net priorities

As you might have guessed I’m revisiting some LXC issues lately, as I said I’m using it in my work here in the US right now, and also because an user asked me about a few things in the past few days.

One of the issues he brought up is that I didn’t bother to update to 0.8.0 yet, which is true because when I briefly tried it from GIT last month it had some issues, and I wanted to wait for the final to be released, but maybe it’s not that good an idea since I hit another issue now, so I’m bumping it as we speak.

Interestingly enough, one feature of the latest kernel (3.3) seems to break with a feature of the new LXC. Since, as I said in the previous post the cgroup feature is getting used in new and new contexts, it seems like LXC finally decided to create hierarchical control groups, akin to what OpenRC and systemd do.

Unfortunately, it seems like you can’t create a sub-group on a netprio control group, which is the new network priority group introduced with kernel 3.3. The previous LXC version (0.7.5) works fine with that network priority group inserted — I think this here is a bug in the Linux kernel though. I just don’t have the time to go fix it now.

Of course this is not all (you wish, huh?). One of the interesting features of LXC allows launching commands within a container, without having to launch the container itself. This is something I wanted to use here, simply because we’ll get lots of containers at that point, most likely. To do this you use the lxc-execute command, which in turn uses a binary called lxc-init to wait for the processes to complete before tearing down the container.

This command uses POSIX Message Queues (Posix MQs) to pass the information around, making it the second project I have ever seen using that stuff. Anyway there is now a check in the ebuild that warns you if you have the feature disabled.

You’d also think that since LXC uses its own lxc-init to run within the container, you’ll get the binary bound by default? Nope, you’ve got to add it yourself. Well, at least it’ll be a static binary that works out of the box? Nope again, the binary is actually linked to the shared liblxc.so.0 shared object. So it’s just a matter of linking it to the static archive instead? Hahaha, we wish.

It seems like IBM (or at least Lezcano who maintains LXC) don’t like libtool, they don’t understand it, and they don’t understand why we would need it (hint: because they don’t understand shared object versioning is already a good answer), so they simply decided to … create only the shared object library by using a dirty hack in automake that was not even supposed to work: the shared object is declared as a PROGRAM target.

Now why would I blame IBM here? People before complained that I shouldn’t be forcing this upon IBM, as Lezcano just happen to work for them. Nope, check the AUTHORS file, it says “IBM Corporation” in there; that means the code is intended to come as a corporate contribution, and the fact that its buildsystem sucks big time is IBM’s own fault!

Would it be easy to fix this issue? Probably yes, but I know that I’ll have to maintain it forever since upstream doesn’t care. I would also probably bet that LXC could start using multi-call binaries rather than the current situation, so that you don’t have internal dependencies at least, as the library is the biggest part of all, and the commands are all at most 10KiB.

Anyway for now there is a patch in the Gentoo ebuild for 0.8.0_rc1-r1, which changes the build system to use libtool, and link lxc-init statically. This is disabled if you turn the vanilla USE flag on, the same way I’ve done before for the init script, since upstream is probably not going to like that.