What happened with pam_namespace?

In the previous post I’ve written the why and how of private temporary directories with the pam_mktemp module. Today I’m going to present you the alternative – pam_namespace – and discuss its problems.

Since all software tries to write to /tmp, the idea implemented by pam_namespace is that of providing a different directory for each user, so you can have private per-user temporary directories without messing with the environment variables, or having to change code. This creates, though, a few problems: some software still uses the temporary directory for multi-user sockets (luckily, nothing major in Gentoo that I know of, PostgreSQL was changed already), but most importantly, the polyinstantiation session that is used by pam_namespace to achieve its results restricts mounts to be available only on that particular session.

What does that mean? Well, let’s say you are logged in as a standard user in the system, you have to mount some partition that is by default unmounted (for security, or because it’s used for backups or stuff like that). To mount it you su to root, and run mount; you check the partition is mounted and it’s okay. You exit from the super user shell and… the mount point is empty. Why? Because the su session is the only one that could have seen that mount. This gets extended to extreme consequences when you use hotplug and automount systems, like HAL, or even more DeviceKit (the latter because it opens a new session for each mount), or when you use sudo mount to try mounting a partition (where the session is opened and closed right away).

So what are our currently available mitigation options? It really depends, because we have no valid “catch-all” option: if you run a desktop system where you don’t need samba, you can have a (light) mitigation by using pam_mktemp; if you run a server system with no hotplug, you might want to try the pam_namespace approach (which isn’t really ready for Gentoo integration yet). What we’d really need is something that uses the pam_namespace approach, but only dissociates the /tmp directory rather than the whole system).

The other alternative, easier, with better results on the long run, and most importantly not tied to Linux, would be to actually make it policy that software in Gentoo works by writing to TMPDIR and not /tmp, and make sure that /tmp cannot be written to at all. Unfortunately this might require some Gentoo-side patching because upstream don’t always accept this request (see my GnuPG notes), and fixes for, at least, Samba to properly change the CWD.

Mitigation of insecure temporary files

Last week, I wrote about my experimentation with pam_namespace to replace pam_mktemp (that breaks samba and does not cover all software). Unfortunately, a couple of days after starting this I gave up on it entirely, I did write down a quick comment about that, and promised to write down what the issue is exactly.

But since I know that people like to have some context on why I’m doing something and why that’s important for users, I’ll first post some summary on what the problem is with insecure temporary files and why it’s useful to mitigate them. And while I’m at it, I’ll explain the drawbacks of my original approach that is currently available through pambase.

Software often have security problems with temporary files, that might lead to information disclosure, or worse . This is caused both by manually-created temporary files, and by old, broken functions like tmpnam() which are deemed insecure. For this reason there are two main movements trying to work on this: from one side, Debian security team has developed a script that can identify a large amount of insecure temporary files creation (tracker from Gentoo); on the other hand, OpenWall developed pam_mktemp (do not confuse it with the pam_mktemp from the Debian-Athena distribution, that used the same name with a different version and with a totally different interface — not a very good choice there), which creates per-user directories, properly set up so that insecure temporary files from an user cannot be exploited from another… that is if the software supports TMPDIR.

Yes because the pam_mktemp module does not rely on anything particular, either from the kernel or from the PAM implementation: it works fine with Linux-PAM and OpenPAM, on Linux and FreeBSD and most likely on any operating system and PAM implementation. What it mostly does is setting up the directory structure, and then sets the TMPDIR environment variable to the new one. All the software knowing about this variable (knowing about it is part of the GNU coding standards), or are using the proper commands and functions to create the temporary files, will be using the user-private temporary directory and this reduces the impact of insecure temporary file creation.

Unfortunately, this does not work as flawlessly as I hoped: first problem is that Samba, for instance, has some problems with the idea of having per-user temporary directories: indeed, after it starts (with user nobody) it moves to the temporary directory as CWD (Current Work Directory) and tries to stay there after changing user to the one you’re authenticating with. Since the directory for nobody is not accessible by the rest of the users, Samba stops working there.

The second problem is more insidious, because while the previous one can be deemed an incompatibility and thus would simply require users to change their setup slightly, this one moots the main reason why we want to use pam_mktemp: vulnerability mitigation. We want to reduce the chance that insecure temporary file creation might be exploited for information disclosure or worse, but if the code is unable to create properly secured temporary files, it’s very likely that it will not respect the TMPDIR setting either.

At the end, what pam_mktemp is actually able to prevent is insecure temporary file creation by mistake in code that is otherwise sound. While this happens, and thus the module is not totally useless, this is quite rare; most of the code that would have this kind of problems would also have to be modified to accept TMPDIR too. And you have to add to this the software that insists on not supporting that setting: GnuPG for instance, where upstream has claimed multiple mirror-climbing reasons for not abiding to the coding standards it should abide to, being a GNU project (just to qualify this statement, one of the reasons I was given is that if they abided to TMPDIR, the user could set that to his home directory, which could be stored on NFS, and that would open a security risk as it would expose the agent to other machines – if you got any idea how local unix sockets work, you know this is not the case since the socket is local to the kernel, and at the same time, such misconfiguration is likely to be a problem for other reasons and something that exists between keyboard and chair).

An alternative to all this is the pam_namespace module, but that also has its share of problems, but those, will be posted tomorrow. (For once, I already have the post written so it will happen! I just wanted to give this drop by drop since people keep saying I write too much).

Masochism never ends — Some improvements for Gentoo safety

Those of you who follow my blog for a long time might have already guessed what I’m going to talk about at this point: PAM. Each and every time I end up working on PAM I remind myself I’m probably a masochist in the deep of my soul, which is pretty bad by itself. I have to clear up though, that I’m not in the same rank as Max Mosley and that my masochism stop at maintaining packages that are ridiculously left alone by anybody else. Just to be safe, okay?

So what’s up in PAM land? Well, there are currently two things I’m experimenting with that I would like to see in our default PAM setup (through pambase) in the next months: capabilities and namespaces.

I already wrote something about capabilities in February 2008, but I never went around making sure it worked like it should. Right now I haven’t really even started the work to support that, but I really intend to get this working before the end of the year. This would complement eventual support for filesystem-based capabilities in Portage that I’ve already proposed a few times before. I got interested again in implementing that after reading that Fedora 12 is also doing so and that it would have solved the recent PulseAudio vulnerability (which luckily we addressed right away in another way).

For what concerns namespaces, instead, this is not only a security issue that I’m investigating but also a possible system setup improvement that could really be something. You might remember that some time ago I made it possible to quickly support pam_mktemp which creates per-user temporary directories in /tmp and make them inaccessible from other users.

Unfortunately there have been, with time, quite a few problems related to that functionality: at the time, start-stop-daemon in OpenRC didn’t really set up environment properly (and I think the same is also true for the version bundled with baselayout1), which in turn created problems to me and others using the daemon mode of emacs because it used the temporary directory to create a listening socket for the client to connect (and daemon and user ended up having different temporary directories). Nowadays, with the working command, and services using pam_mktemp, the problem appears with Samba: when starting Samba on a system set up with pam_mktemp, logging into shares that have usernames and passwords fail because Samba tries to use the user-defined temporary directory as the user “nobody” (which does not have the privilege to access that). Finally, lots of packages simply ignore the TMPDIR variable, either because they “want consistency” or because, like it’s the case for gnupg, upstream has no clue about Unix sockets and expects TMPDIR to mock stuff up via network, thus ignores it (breaching the GNU coding style guidelines, by the way).

So I decided to go with the alternative way, which is not supported on non-Linux systems (thus pam_mktemp is not going to die just yet), but is also available off the shelf on the Linux-PAM package: pam_namespace and polyinstantiated directories. To cut the story short and make it understandable to users, it simply means that once it’s set up, /tmp as seen by different users is a different directory altogether.

I’m still not sure if this is the way to go to be honest. When I first looked into it, it would break postgres, because the older settings would use /tmp to store the connection socket rather than /var/run; I also am not sure whether it’s totally fine in respect to security: since I usually start my services sudo’ing from my user, the directories used for polyinstantiation have “flame” as group; of course I cannot access them because I cannot access the base directory, but still… Also, this method seems to break the tramp feature of Emacs that allows to edit root-owned files from within an user session (it assumes that /tmp is always the same between users afaics).

Now, I said that namespaces are not only a security feature but can also help with configuration; some time ago I tried finding an easy way to set the XDG_CACHE_DIR variable for users so that instead of saving in ~/.cache it would use /var/cache (reason for this is that the home file system is usually smaller and should not be wasted for cache – in my case it’s also under RAID – while the /var/cache file system can just be cleaned up as needed). Unfortunately, our env-update command does not take it gently if hte $USER variable is used to define another variable, so I gave up. With pam_namespace, it would be possible to have a /var/cache/user directory that is an instance of /var/cache/users/$USER, and then have XDG_CACHE_DIR to use that directory. Similarly we could do that for /var/run/user so that local instance sockets are saved there rather than in /tmp (or /home) where they don’t belong.

So this is what is boiling in the PAM pot for the next months; you can always speed it up at your pleasure.

Security mitigation strategies or, the only secure computer is the one yet to assemble.

Short preamble: I’m in a very depressed mood, like I haven’t been in month; this is very bad for my health but usually means I can focus on things much better, so you might actually find out I’m doing more than usual. Of course there is also to count in that I’m working during holidays so it’s not going to be all nice at all, even counting my depression off.

As I’ve written, I don’t trust closed-source software even the slightest; even though it does not really mean that free software is much better, process-wise, dealing with bundled libraries (like the bundled libs bug shows), with free software, or at least open-source software, there is the chance to check the sources out to fix the eventual issues.

This means that I won’t be using closed source software where security is a major concern, but since sometimes I have to use closed-source software, like Skype, or Sun’s compiler, it’s obvious that I have to find a compromise so I can still use them and yet feel reasonably safe. This is what is usually called having a mitigation strategy.

One of the most complex and well known mitigation strategies is of course SElinux, which makes a Linux system more like an APC than a computer. But such a system is probably safe to consider overkill for most systems, especially power user desktop systems.

Since this is, as I said, overkill, I’m more prone to look at smaller strategies, one of which I already discussed about: pam_mktemp . This module allows to create per-user private directories that make it much harder to exploit insecure temporary files vulnerabilities. Which is very nice since this seems to be a very common class of vulnerabilities, and my data shows that there is way too much software that still uses insecure functions to create temporary files, closed and open source alike.

Unfortunately, as you can read in my earlier blog post, this is not automatically a way out of the problem. The start-stop-daemon command from OpenRC plays nice with this just in the last release, and even with that, there are problems. The first problem is that the way pam_mktemp works, there is a need for the software calling PAM to open the session to properly set up the environment with its changes (which is what s-s-d lacked in previous versions). This causes for instance the gnome-keyring daemon to start with the wrong temporary directory when started by the PAM session chain. Even though pam_mktemp is invoked before the daemon, by the time it’s started the TMPDIR variable is not set in the environment. The reason for this is that the variable should not be changed if the session chain aborts the login.

The second problem is that not all software supports TMPDIR properly; Emacs has been fixed recently and now the emacs daemon starts up properly, but other software ignores TMPDIR altogether. VirtualBox (of which I still have things to say beside this) does not respect it for instance, which means that the module wouldn’t have spared you from the recent vulnerability that involved the software.

The third problem is that sometimes software expects TMPDIR to be world-readable, which is a bad assumption; Samba does this, and since s-s-d is now fixed, it now fails to work on my system. I still haven’t found out whether the PAM session chain was called at that point, and it’s just duplicating the problem with s-s-d with a different symptom, or if it fails to call it entirely. In either case, it’s a thing that has to be fixed to make sure that mitigation strategies like this one get in the default spirit of users.

But again this is just one part of the problem, and one part of mitigation. Other problems relate to the way we run some of the services, a lot of which still run as root rather than under a unprivileged user; while the git-daemon issue is now solved and the default install does not run as root any longer, there are more daemons that have the same problems.

Just as an example, I noticed that the iSCSI daemon ietd still runs under root, and I’ve added that to the list of software I have to check to see if I can improve it. Similarly, the init script for mpd does not use s-s-d to switch user but leaves it to mpd itself, spawning it by default with unneeded root privileges, and additionally not allowing pam_mktemp to create a new temporary directory for the mpd user (I have to spend some time on that since I’d also like to provide an alternative init script with multiplexing, which would then allow to run multiple mpds for different users, and in my case to just have the single mpd running as my own user rather than a different user entirely).

At any rate, I’m going to continue my best to make sure that secure defaults are in place in Gentoo, and that further mitigation strategies can be made available so that the users forced to use proprietary closed-source software don’t need to just accept whatever comes their way. Please join my efforts, if you can, by checking which software ignores TMPDIR and asking nicely upstream to fix the issue.

Private temporary directories

One more addition to pambase has been the mktemp USE flag. With that flag enabled, the default system-auth stack features pam_mktemp module, again from OpenWall.

All this module does is setting up a per-user private temporary directory inside /tmp, and set TMP and TMPDIR environment variables so that the programs (at least those respecting them) use the new private temporary directory rather than the globally-accessible one.

This is useful as a mitigation strategy for temporary file attacks (race issues and symlink attacks); while it does not magically fix them, and their seriousness is not to be ignored, it might make it more difficult to exploit them. You can think of it to something alike to PaX, hardened and SElinux.

There are, though, a few thing that need to be improved for it to be totally useful; and until those are completed, and the setup widely tested, it will not be enabled by default.

To begin with, not all applications make use of TMPDIR, I already blogged about sqlite ignoring TMPDIR entirely, but there are other cases like that, for instance, Emacs’s server support hardcodes /tmp as directory (I’ve mailed emacs-devel and they are now discussing about using TMPDIR as every other application), as well as gpg-agent and ssh-agent. I wonder how many applications will have broken designs in that regard; I know ESounD requires /tmp as path for its socket, so PulseAudio will not respect TMPDIR by default; Xorg also seems to use /tmp for locks (although I would have expected them to be in /var/lock), and PostgreSQL uses /tmp for its sockets (why not /var/run?).

Another issue is that pam_mktemp seems to handle in specific ways ext2/ext3, with its “appendable” flag, but the same does not hold true for other filesystems (I admit I don’t know how that works, and if other filesystems support them; I’m quite sure BSDs support that though – too bad they didn’t port pam_mktemp as well as pam_passwdqc). Some porting of this would probably be useful, I suppose OpenWall has a default setup where it does not make sense to check for more than ext3. SoC, anyone? ;)

Then there is the issue to make sure that setuid and setgid programs all work fine, it probably will require quite a bit of work though. I suppose this could get well along with file-based capabilities support, as that would limit a lot the amount of setgid and setuid programs, they’d just need capabilities to access the minimum privileges they need to.

But that’s a story for a different day…

A different way to achieve this is to use pam_namespace (provided by Linux-PAM), which allows to create per-user instances of any directory on the filesystem. This would then create a different /tmp directory without fiddling with TMPDIR. While this is quite interesting as it allows to workaround the broken applications hardcoding /tmp into their source, it would probably break Xorg, PostgreSQL and ESounD as they’ll get a different per-user directory too.

On the other hand, if we’re able to get all the software fixed in its design, so that global data (locks and sockets) goes to /var, then it might work fine in the future. It would also allow for a more graceful handling of FreeDesktop-defined cache directories (XDG_DATA_DIRS) as you could set the environment variable for all the users to, say, /var/cache/xdg and then instantiate a different version of that per-user.

Right now, though, I prefer the pam_mktemp solution, even if it will require more fixes around so that /tmp is not used.