I was curious to see why there was much noise coming from the Gentoo Wiki LXC page to my blog lately, so I decided to peek at the documentation there once again, slightly more thoroughly; while there are a number of things that could likely be simplified (but unless I’m paid to do so I’m not going to work on documenting LXC until it’s stable enough upstream; as it is it might just be a waste of time since they change the format of configuration files every other release), there is one thing that really baffled me:
If you want to share distfiles from your host, you might want to look at unionfs, which lets you mount the files from your host’s /usr/portage/distfiles/ read-only whilst enabling read-write within the guest.
While unionfs is the kind of feature that goes quite well with LXC, for something as simple as that is quite an overkill. On the other hand I remembered that I also had to ask about this to Zac, so it might be a good idea to document this a moment.
Portage already allows you to share a read-only distfiles directory.
You probably remember that I originally started using Kerberos because I wanted some safe way to share a few directories between Yamato and the other boxes. One of the common situations I had was the problem of sharing the distfiles, since quite a few are big enough that I’d rather not download them twice. Well, even with that implemented, I couldn’t get Portage to properly access them, so I looked for alternative ways. And there is a variable that solves everything: PORTAGE_RO_DISTDIRS.
How does that work? Simply mount some path such as /var/cache/portage/distfiles/remote as the remote distfiles directory, then set PORTAGE_RO_DISTDIRS to that value. If the required files are in the read-only directory, and not in the read-write one, Portage will symlink them and ensure access on a stable path. Depending on your setup, you might want to export/import the directory as NFS, or – in case of LXC – you might want to set it up as a read-only bind mount. Whatever you choose, you don’t need kernel support for unionfs to use a read-only distfiles directory. Okay?
Hello,I wanted to go further and I shared the whole /usr/portage directory and add FEATURES=”buildpkg” to the make.conf. (Is it really ok to do that ?) So I can share the whole portage tree and the binaries produced by emerge.But the main problem is that all the containers have to share the same useflags. It is generally good but in a small amount of cases this is not.For example all my containers use MySQL package with the ‘minimal’ useflag but one is the MySQL server so I want to set ‘-minimal’.It’d be cool if when compiling portage uses the binary package only if it is build with useflags requested by the system.And that would be awesome if when portage build a package with different useflags it stores the new one with others binaries packages but without overwrite the previous one (same package with other useflags).I’ve not seen anything in the documentation about this two points. Am I missing something ? Is it possible ?Thanks.
Yes and no… I know there should be nowadays support for Portage to only use a binary package if USE flags are the one requested, but the fallback is to build it from scratch in that case.As to store different packages with different USE flag settings, I’m pretty sure somebody was working on that, but on how the situation is nowadays, no clue.
Actually, the PORTAGE_RO_DISTDIRS feature is not really needed: You could as well setup some script as FETCHCOMMAND which symlinks/moves/whatever you want and only in case of failure attempts wget (or however you want to download the file). Of course, if plain symlinking is enough for your purpose, the PORTAGE_RO_DISTDIRS feature is simpler to setup (and perhaps will be a little bit quicker).
Uhm, between writing a script to do a number of things, and just leaving @FETCHCOMMAND@ alone, setting up @PORTAGE_RO_DISTDIRS@ and letting it take care of it, I’m pretty sure my solution is much simpler…
I didn’t claim the contrary:> Of course, if plain symlinking is enough for your purpose, the PORTAGE_RO_DISTDIRS feature is simpler to setupHowever, FETCHCOMMAND was available since many years while from your blog it appeared that sharing read-only directories is only possible since the last few releases of portage.
Hey Diego, I am the author of the original comment.The PORTAGE_RO_DISTDIRS feature based solution is certainly a good one.Unfortunately, it’s exactly the sort of thing that non Gentoo developers (or at least, people who don’t casually “man make.conf” under Gentoo) are likely to miss.Forgetting Gentoo specifics for a minute, more broadly under Unix both union mounts and RAM based filesystems such as tmpfs are very useful within read only environments such as ro-root PXE boots, certain containers, etc. I do think throwing people a mental hook to some of the options there is probably helpful.Based on the PORTAGE_RO_DISTDIRS comments that appeared in the Wiki I opened an issue on lxc-gentoo to remind myself to add support in the future a few days ago.In other news I added an aggressive capability drop to lxc-gentoo a few days ago… it feels like the LXC model for this (explicitly drop rather than drop all minus an explicit subset to maintain) is fundamentally the wrong one. I’ve been meaning to open a bug with those guys and just got around to it now: https://sourceforge.net/tra…