So I finally added lxc 0.6.5 to the tree for all of the Gentoo users to try it if they care; on the bright side, the lxc.mount option seem to finally work fine, and I also found the problem I complained about a few days ago. It is this latter problem that made me feel like writing this half-rant post.
Half-rant because I can see they are going to great extents to improve it from release to release, but still a rant because I think it should have been less chaotic to begin with. On the other hand, I could probably be more constructive if I went on to provide patches… I’ll probably do that in the future if I free myself, but if you follow my blog you know I’m quite swamped already between different things.
Starting from the 0.6.4 release, they dropped the “initialisation” task, and just let you run lxc-start with the config file. It was definitely a nice way to do it, as the init command wasn’t doing anything heavy that shouldn’t be done on first startup anyway. It was, though, a bit of a problem for scripts that used the old method as the simple lxc-start -n myguest (the init step wouldn’t be needed after a restart of the host) would mess up your system badly, as it would spawn a new container using / as the new root… overriding your own system bootup. Thankfully, Andrian quickly added a check that refused to proceed if a configuration file was not given. This does not save you from being able to explicitly mess your system up by using / as your new root, but at least avoids possible mistakes when using the old-style calls.
So what about the 0.6.5 problem? Well the problem came to be because 0.6.5 actually implements a nice feature (contributed by a non-core developer it seems): root pivoting. The idea is to drop access to the old root, so that the guest cannot in any way access the host’s filesystem unless given access to. It’s a very good idea, but there are two problems with it: it doesn’t really do it systematically, but rather with a “try and hope” approach, and it failed under certain conditions, saying that the original root is still busy (note here, since this happens within the cgroup’s mount namespace, it doesn’t matter to the rest of the system).
At the end, last night I was able to identify the problem: I had this line in the fstab file used by lxc itself:
none /tmp tmpfs size=200m 0 0
What’s wrong with it? The mountpoint. The fstab (and lxc.mount commands) are used without previous validation or handling, so this is not mounting the /tmp for the guest, but the /tmp for the host, within the guest’s mount namespace. The result is that /tmp gets mounted twice (once inherited by the base mount namespace, once within the guest’s namespace, but it’s only unmounted once (as the unmount list keeps each mount point exactly once). This is quite an obvious error on my part, I should have used /media/chroots/tinderbox/tmp as mountpoint, but LXC being unable to catch the mistake in mountpoint (at least warning about it) is a definite problem.
Another thing that makes me feel like LXC really needs some polishing is that you cannot just run the commands from the source directory: the build system uses autoconf and automake, but the authors explicitly backed away from libtool as it’s “Linux-only” (which really doesn’t say much about the usefulness of libtool in this case). given I’m not even sure whether the liblxc library is supposed to be ABI stable or not (they have never bumped the soname, but that is suspicious), it might really be better if they used libtool and learnt out to handle it. Also, it uses badly recursive Makefiles, it would probably take just a second to build if I remade the build system as a standard non-recursive autotools package, like udev.
Oh well, let’s hope for the future releases to improve polishing, bit by bit!
