Tag: OpenSSH

Windows 10, OpenSSH and YubiKey

Update 2020-08-23: I found WinCryptSSH by chance, and that seems to take care of having an actual agent system set up as well, so that this works with WSL! Give that a try, instead of following the advice on most of this post! You can still read it for context, though.

You may remember that a few months ago I suggested that Windows 10 is an interesting FLOSS development platform now, and that I decided to start using Windows 10 on my Dell XPS laptop (also in the hope that the problem I had with the battery would be caused by Linux — and the answer to that is “nope”, indeed the laptop’s battery is terrible.) One of the things I realised setting all of those up, is that I found myself unable to use my usual OpenPGP-based token, and I thought I would try using a YubiKey 5 instead.

Now, between me and Yubico there’s not much love lost, but I thought I would try to make my life easier by using a smartcard that seemed to have a company interested in this kind of usage behind it. Turns out that this was only partially working, unfortunately.

The plan was to set up the PIV mode of the YubiKey 5 to provide the authentication certificate, rather than trying to use the OpenPGP mode. The reason for that is to be found on Yubico’s own website:

GPG4Win’s smart card support is not rock solid; occasionally you might get error messages when trying to access the YubiKey. It might happen after removing and re-inserting the YubiKey, or after your computer has been in sleep mode, etc. This can be resolved by restarting gpg-agent [snip]

Given that GnuPG’s own smartcard support is kind of terrible already, and not wanting to get into the yak shaving of getting that to work on Windows, I was hoping that using the more common (on Windows) interface of PKCS#11, which OpenSSH supports natively (sort of). To give a very quick and oversimplified summary, PKCS#11 is the definition of an API/ABI that end user software, such as OpenSSH, can use to interface with middleware that provides access to PKI-related functions. Many smartcard manufacturers provide ready made middleware implementing a PKCS#11 interface, which I thought Windows supported directly, but I may be wrong. Mozilla browsers rely on this particular interface to handle CA certificates as well, to the point that the NSS library that Mozilla uses is pretty much a two-part component with a PKCS#11 provider and a PKCS#11 client.

As it turns out, Yubico develops a PKCS#11 middleware for YubiKey as part of yubiko-piv-tool, and provides documentation on how to use it for SSH authentication. Unfortunately the instructions don’t really expand to including needed information for using this on Windows, as they explicitly say at the top of the page. But that would never stop me, after all. Most of the setup described in that document is perfectly applicable to Windows, by the way — until you get to the first issue…

The first issue with setting this up is that while Windows 10 does ship with OpenSSH client (and server), it does not ship with PKCS#11 support enabled. Indeed, the version provided even with 20H1 (the current most recent non-Insider build) is 7.7p1, while the current upstream release would be 8.3p1. Thankfully, Microsoft is providing a more up to date build, although that’s also still blocked at 8.1p1. The important part is that these binaries do include PKCS#11 support.

For this whole to work, you need to have both the OpenSSH binaries provided by Microsoft, and the Yubico libraries (DLL) in folders that are part of the PATH environment variable. And they also need to match the ABI. So if you’re setting this up on an x64 system, and used the 64-bit OpenSSH build, you should install the 64-bit Yubico PIV Tool, and vice-versa for 32-bit installs.

Now, despite the installer warning you that to use the PKCS#11 provider you need to have the bin folder in the PATH variable, and that loading the provider will full path will not be enough… the installer does not offer to modify the PATH itself, unlike the Git installer that does, to make it easy to use globally. This is not too terrible, because you also need to add the new OpenSSH in the PATH. For myself, I decided to use a simple OpenSSH folder in my home.

Modifying the environment variables in (English) Windows 10 is fairly straightforward: hit the Search function, and type Environment — it’ll come up with the right control panel, and you can then edit the PATH variable and just browse for the right folder.

There is one more thing you need to do, and that is to create a .ssh/config file in your home directory with the following content:

PKCS11Provider libykcs11.dll

This instructs OpenSSH to look for the Yubico PKCS#11 provider automatically instead of having to specify it on the command line. Note once again that while you could provide the full path to the DLL file, if you didn’t add it to the PATH, it would likely not load — Windows 10 is stricter in where to look for dependencies when dynamically loading a DLL. And also, you’ll get a “not a valid win32 application” error if you installed/configured the wrong version of the Yubico tool (32-bit vs 64-bit).

After that is done, ta-dah! It should work fine!

Screenshot of Windows PowerShell using a YubiKey 5 to authenticate to a Gentoo Linux system.

This works, when using PowerShell. You get asked to enter the PIN for the YubiKey, and you login just fine. Working exactly as intended there.

Unfortunately, the next step I wanted to use this for is to use VSCode to connect to my NUC, and work on things like usbmon-tools remotely, so for that to work, I needed to be able to use this authentication method through the Visual Studio Code remote host mode… and that’s not working at the time of writing. The prompt comes up, but VSCode does not appear to proxy it to anything into its UI for me to answer it.

I’m surprised, because as far as I can tell, the code responsible for the prompt uses the correct read_passphrase() function call for it to be a prompt proxied to the askpass implementation, which I thought was already taken care of by VSCode. I have not spent too much time debugging this problem yet, but if someone is more familiar than me with VSCode and can track down what should happen there, I’d be very happy to hear about it. For now, I filed an issue.

Update 2020-08-04: Rob Lourens from Microsoft moved the issue to the the right repository and pointed to another issue (filed later but in the right place).

The workaround to use this from VSCode, it’s to make sure that "remote.SSH.useLocalServer": true is set, and click on the Details link at the bottom-right corner when it’s trying to connect, to type in the PIN. At which point everything seem to work fine, and even use the connection multiplexer to avoid requesting it all the time.

Screenshot of Visual Studio Code showing a remote SSH connection to my Linux NUC with usbmon-tool open.

GnuPG Agent Forwarding with OpenPGP cards

Finally, after many months (a year?) absence, I’m officially back as a Gentoo Linux developer with proper tree access. I have not used my powers much yet, but I wanted to at least point out why it took me so long to make it possible for me to come back.

There are two main obstacles that I was facing, the first was that the manifest signing key needed to be replaced for a number of reasons, and I had no easy access to the smartcard with my main key which I’ve been using since 2010. Instead I set myself up with a separate key on a “token”: a SIM-sized OpenPGP card installed into a Gemalto fixed-card reader (IDBridge K30.) Unfortunately this key was not cross-signed (and still isn’t, but we’re fixing that.)

The other problem is that for many (yet not all) packages I worked on, I would work on a remote system, one of the containers in my “testing server”, which also host(ed) the tinderbox. This means that the signing needs to happen on the remote host, although the key cannot leave the smartcard on the local laptop. GPG forwarding is not very simple but it has sort-of-recently become possible without too much intrusion.

The first thing to know is that you really want GnuPG 2.1; this is because it makes your life significantly easier as the key management is handed over to the Agent in all cases, which means there is no need for the “stubs” of the private key to be generated in the remote home. The other improvement in GnuPG 2.1 is that there is better sockets’ handling: on systemd it uses the /run/user path, and in general it uses a standard named socket with no way to opt-out. It also allows you to define an extra socket that is allowed to issue signature requests, but not modify the card or secret keys, which is part of the defence in depth when allowing remote access to the key.

There are instructions which should make it easier to set up, but they don’t quite work the way I read them, in particular because they require a separate wrapper to set up the connection. Instead, together with Robin we managed to figure out how to make this work correctly with GnuPG 2.0. Of course, since that Sunday, GnuPG 2.1 was made stable, and so it stopped working, too.

So, without further ado, let’s see what is needed to get this to work correctly. In the following example we assume we have two hosts, “local” and “remote”; we’ll have to change ~/.gnupg/gpg-agent.conf and ~/.ssh/config on “local”, and /etc/ssh/sshd_config on “remote”.

The first step is to ask GPG Agent to listen to an “extra socket”, which is the restricted socket that we want to forward. We also want for it to keep the display information in memory, I’ll get to explain that towards the end.

# local:~/.gnupg/gpg-agent.conf

keep-display
extra-socket ~/.gnupg/S.gpg-agent.remote

This is particularly important for systemd users because the normal sockets would be in /run and so it’s a bit more complicated to forward them correctly.

Secondly, we need to ask OpenSSH to forward this Unix socket to the remote host; for this to work you need at least OpenSSH 6.7, but since that’s now quite old, we can be mostly safe to assume you are using that. Unlike GnuPG, SSH does not correctly expand tilde for home, so you’ll have to know the actual paths we want to write the unix at the right path.

# local:~/.ssh/config

Host remote
RemoteForward /home/remote-user/.gnupg/S.gpg-agent /home/local-user/.gnupg/S.gpg-agent.remote
ExitOnForwardFailure yes

Note that the paths need to be fully qualified and are in the order remote, local. The ExitOnForwardFailure option ensures that you don’t get a silent failure to listen to the socket and fight for an hour trying to figure out what’s going on. Yes, I had that problem. By the way, you can combine this just fine with the now not so unknown SSH tricks I spoke about nearly six years ago.

Now is the slightly trickier part. Unlike the original gpg-agent, OpenSSH will not clean up the socket when it’s closed, which means you need to make sure it gets overwritten. This is indeed the main logic behind the remote-gpg script that I linked earlier, and the reason for that is that the StreamLocalBindUnlink option, which seems like the most obvious parameter to set, does not behave like most people would expect it to.

The explanation for that is actually simple: as the name of the option says, this only works for local sockets. So if you’re using the LocalForward it works exactly as intended, but if you’re using RemoteForward (as we need in this case), the one on the client side is just going to be thoroughly ignored. Which means you need to do this instead:

# remote:/etc/sshd/config

StreamLocalBindUnlink yes

Note that this applies to all the requests. You could reduce the possibility of bugs by using the Match directive to reduce them to the single user you care about, but that’s left up to you as an exercise.

At this point, things should just work: GnuPG 2.1 will notice there is a socket already so it will not start up a new gpg-agent process, and it will still start up every other project that is needed. And since as I said the stubs are not needed, there is no need to use --card-edit or --card-status (which, by the way, would not be working anyway as they are forbidden by the extra socket.)

However, if you try at this point to sign anything, it’ll just fail because it does not know anything about the key; so before you use it, you need to fetch a copy of the public key for the key id you want to use:

gpg --recv-key ${yourkeyid}
gpg -u ${yourkeyid} --clearsign --stdin

(It will also work without -u if that’s the only key it knows about.)

So what is about keep-display in local:~/.gnupg/gpg-agent.conf? One of the issues I faced with Robin was gpg failing saying something about “file not found”, though obviously the file I was using was found. A bit of fiddling later found these problems:

  • before GnuPG 2.1 I would start up gpg-agent with the wrapper script I wrote, and so it would usually be started by one of my Konsole session;
  • most of the time the Konsole session with the agent would be dead by the time I went to SSH;
  • the PIN for the card has to be typed on the local machine, not remote, so the pinentry binary should always be started locally; but it would get (some of) the environment variables from the session in which gpg is running, which means the shell on “remote”;
  • using DISPLAY=:0 gpg would make it work fine as pinentry would be told to open the local display.

A bit of sniffing around the source code brought up that keep-display option, which essentially tells pinentry to ignore the session where gpg is running and only consider the DISPLAY variable when gpg-agent is started. This works for me, but it has a few drawbacks. It would not work correctly if you tried to use GnuPG out of the X11 session, and it would not work correctly if you have multiple X11 sessions (say through X11 forwarding.) I think this is fine.

There is another general drawback on this solution: if two clients connect to the same SSH server with the same user, the last one connecting is the one that actually gets to provide its gpg-agent. The other one will be silently overruled. I”m afraid there is no obvious way to fix this. The way OpenSSH itself handles this for the SSH Agent forwarding is to provide a randomly-named socket in /tmp, and set the environment variable to point at it. This would not work for GnuPG anymore because it now standardised the socket name, and removed support for passing it in environment variables.

Asynchronous Munin

If you’re a Munin user in Gentoo and you look at ChangeLogs you probably noticed that yesterday I did commit quite a few changes to the latest ~arch ebuild of it. The main topic for these changes was async support, which unfortunately I think is still not ready yet, but let’s take a step back. Munin 2.0 brought one feature that was clamored for, and one that was simply extremely interesting: the former is the native SSH transport, the others is what is called “Asynchronous Nodes”.

On a classic node whenever you’re running the update, you actually have to connect to each monitored node (real or virtual), get the list of plugins, get the config of each plugin (which is not cached by the node), and then get the data for said plugin. For things that are easy to get because they only require you to get data out of a file, this is okay, but when you have to actually contact services that take time to respond, it’s a huge pain in the neck. This gets even worse when SNMP is involved, because then you have to actually make multiple requests (for multiple values) both to get the configuration, and to get the values.

To the mix you have to add that the default timeout on the node, for various reason, is 10 seconds which, as I wrote before makes it impossible to use the original IPMI plugin for most of the servers available out there (my plugin instead seem to work just fine, thanks to FreeIPMI). You can increase the timeout, even though this is not really documented to begin with (unfortunately like most of the things about Munin) but that does not help in many cases.

So here’s how the Asynchronous node should solve this issue: on a standard node, the requests to the single node are serialized so you’re actually waiting for each to complete before the next one is fetched, as I said, and since this can make the connection to the node take, all in all, a few minutes, and if the connection is severed then, you lose your data. The Asynchronous node, instead, has a different service polling the actual node on the same host, and saves the data in its spool file. The master in this case connects via SSH (it could theoretically work using xinetd but neither me nor Steve care about that), launches the asynchronous client, and then requests all the data that was fetched since the last request.

This has two side-effects: the first is that your foreign network connection is much faster (there is no waiting for the plugins to config and fetch the data), which in turn means that the overall munin-update transaction is faster, but also, if for whatever reason the connection fails at one point (a VPN connection crashes, a network cable is unplugged, …), the spooled data will cover the time that the network was unreachable as well, removing the “holes” in the monitoring that I’ve been seeing way too often lately. The second side effect is that you can actually spool data every five minutes, but only request it every, let’s say, 15, for hosts which does not require constant monitoring, even though you want to keep granularity.

Unfortunately, the async support is not as tested as it should be and there are quite a few things that are not ironed out yet, which is why the support for it in the ebuild has been this much in flux up to this point. Some things have been changed upstream as well: before, you had only one user, and that was used for both the SSH connections and for the plugins to fetch data — unfortunately one of the side effect of this is that you might have given your munin user more access (usually read-only, but often times there’s no way to ensure that’s the case!) to devices, configurations or things like that… and you definitely don’t want to allow direct access to said user. Now we have two users, munin and munin-async, and the latter needs to have an actual shell.

I tried toying with the idea of using the munin-async client as a shell, but the problem is that there are no ways to pass options to it that way so you can’t use --spoolfetch which makes it vastly useless. On the other hand, I was able to get the SSH support a bit more reliable without having to handle configuration files on the Gentoo side (so that it works for other distributions as well, I need that because I have a few CentOS servers at this point), including the ability to use this without requiring netcat on the other side of the SSH connection (using one old trick with OpenSSH). But this is not yet ready, it’ll have to wait for a little longer.

Anyway as usual you can expect updates to the Munin page on the Gentoo Wiki when the new code is fully deployed. The big problem I’m having right now is making sure I don’t screw up with the work’s monitors while I’m playing with improving and fixing Munin itself.

Mostly unknown OpenSSH tricks

I’m not really keen on writing “tricks” kind of posts for software, especially widely known as OpenSSH, but this stuff tends to be quite intriguing now – to me at least – and I think most people wouldn’t know about that.

If you’ve used cvs, svn or other SSH-connected services in the recent past you probably know that one of the most time-wasting tasks for these systems is to connect the ssh session itself. Thankfully, quite a bit of time ago, OpenSSH introduced the “master” connection feature: the first ssh connection you open to a host (with a given username) creates a Unix socket descriptor, which is used by the following sessions. This was really a godsend for when committing a huge number of packages more or less at the same time, when I was still in the KDE team.

Unfortunately, this still required the first connection to be a persistent one; to solve that, I used to start a screen session that connected a few ssh connections, after asking me for the key passphrase. This didn’t make it any nice on the system or on the security, among other things. But fear not, OpenSSH 5.6 brought magic into the mix!

The new ControlPersist option allows for the master connection to be created in a detached SSH process when first connecting to a box, so that there is no need to preventively prepare for processes to be kept around. Basically all you have to do to make use of master connections now is something like this in your ~/.ssh/config:

Host *.gentoo.org
ControlMaster auto
ControlPersist yes

and you’re set: ssh takes care of creating the first master connection if not present, and to delete and recreate it if it’s dropped for whatever reason; you can otherwise force the connection to be closed by using ssh $host -O exit. Lovely!

There is one more trick that I wish to share though, although this time around I’m not sure which OpenSSH version introduced it. You can have from time to time the need to connect to a box that is behind a not-too-hostile firewall, which you have also access to. This is the case at a customer’s of mine where only the main router has a (dynamic) IPv6 address and I have to go through that to connect to the other boxes. The usual trick to follow in such a situation is to use the ProxyCommand option, using ssh and nc (or in my case nc6) to get a raw connection to the other side. I was always a bit bothered by having to do it this way to be honest. Once again, recent versions of OpenSSH solve the problem with the -W option:

Host router.mycustomer.example.com
ProxyCommand none

Host *.mycustomer.example.com
ProxyCommand ssh router.mycustomer.example.com -W %h:%p

With this method, it will be the ssh on my customer’s router to take care of connecting to the box further down the net and redirect that to the input/output streams, without the need for a nc process to be spawned. While this doesn’t look like a biggie, it’s still one less package I have to keep installed on the system, among other things.

What’s the plan with PAM?

Okay after quite a few days working on pambase, and a few discussions, I think I have a decent plan going forward to implement the new pambase. You can find the current status of the code in the repository but of course it is not really enough to get to test it; I’m still undecided whether I should add an overlay to the same repository, or get a PAM-only overlay where I can dump all the ebuilds for review.

So here is the rundown. First of all, as I said many times by now, I’ve replaced the C preprocessor with the M4 tool instead; this provides me a much more complete macro language, which means I can easily add further support, and mess around to disable/enable options as needed. Thanks to this, we now have a number of other authentication schemes, such as LDAP, PKCS#11 (via pam_pkcs11, I’m also going to add pam_p11 which is definitely simpler to set up even though it has less features).

But the main difference is in the services that it provides out of the box; there are now a number of new service files that will be used directly or indirectly by all the packages using PAM in the tree; while I didn’t like increasing the number of service files, there are slight differences in the behaviour of each that makes it necessary to split them around. For instance a remote login by default will not force root to login on a secure tty (although you might want to do that!), and as I said before you cannot easily use the same auth chain for both a login and a password-changer.

Another issue that I’m having trouble wrapping my head around is that you really cannot use the same authentication schemes for both interactive and automatic services; so for instance if you’re logging into a remote mail server you cannot provide an access token to do the login (since it requires it to be connected locally on the server). From one side, the same goes for ssh actually… I guess the only reason why I don’t feel as compelled to tackle it there is that I don’t use PAM for authenticating on SSH and neither should you in most cases.

What is now available in the repository is mostly working fine; although it remains a problem of interfacing it properly: I’m still unsure of the way the various modules stack up. For what it’s worth, one of the most obnoxious things I can think of is properly supporting pam_loginuid: it should be used by services both interactive and non-interactive to properly set auditing to log the user who’s acting (so even if there is privilege escalation you can track down who exploited it), but it should thus not be used by either sudo nor su, nor by things like PostgreSQL or Drizzle.

Right now what we’re going to have for certain are these services:

  • system-local-login (name kept for compatibility) will be used basically only by login — I’m actually tempted to provide /etc/pam.d/login directly as part of pambase, given that it is, yes, provided by shadow right now, but it’s also used by Busybox;
  • system-graphical-login will be used by all the graphical login managers: GDM, XDM, KDM, Slim, Qingy… the idea behind this one is that it is, by default, quiet; unlike what I originally planned, and what GDM now does, this one will not avoid running pam_mail, pam_lastlog and so on so forth; they will be, instead, put into their silent modes: the data will be updated, the variables will be set, but no output will come from them; unfortunately things like Qingy will cause double-setting of the same variables; that’s a problem for another time;
  • system-remote-login is the one service used by sshd (and theoretically by rsh/@rlogin@ if they made sense to be used);
  • system-services is the one used by the various cron daemons, atd, start-stop-daemon and so on: they have no authentication done, but they do have account validation and, most importantly, session support; as I said above, these should set the login uid to ensure that the audit framework knows who to blame for problems in here;
  • system-password is the service used by passwd and the other password changing tools; it only deals with the Unix stack and with (eventually) Gnome-Keyring;
  • system-auth-backends and system-login-backends are the two that give me more trouble to deal with; they have the actual calls to the backends used for authenticating; they are separate so that we can actually have them set up for optionality, so that only one is needed to succeed to allow the user to authenticate to the system, by using the substack option on the previous service; beside substack, the only other solution would have been to use the skip method I’ve been using up to now, and that I haven’t entirely stopped considering to be honest.

Also, among other things, I’ve removed the nullok option from the authentication and session support for pam_unix; this basically means that you can no longer have valid accounts without a password set; the idea is that this will not cause trouble with autologin features, but I’ll cross that bridge in due time. For now it should be enough to have an idea that the code is out there and how should be more or less be used.

Comments are, as usual, quite welcome.

Hardware signatures

If you read Planet Debian as well as this blog, you probably have noticed the number of Debian developers that changed their keys recently, after the shadows cast over the SHA-1 hash algorithm. It is debatable on whether this is an issue now or not, but that’s not what I want to discuss.

There are quite a few reasons why Debian developers are more interested in this than Gentoo developers; while we also sign manifests, there are quite a few things that don’t work that well in our security infrastructure, which we should probably pay more attention to (but I don’t want to digress now), so I don’t blame their consideration of tighter security.

I’m also considering the switch; while I have my key for quite a while, there are a few issues with it: it’s not signed by any Gentoo developer (I actually don’t think I have met anybody in person to be able to exchange documents and stuff), the Manifest signing key is not a subkey of my actual primary key (which by the way contains lots of data of my previous “personas” that don’t matter any longer), and so on so forth. Revoking this all and starting anew might be a good solution.

But, before proceeding, I want finally go get over with the thing and move to hardware cryptography if possible; I already expressed the interest before, but I never dug enough to find the important information, now I’m looking for that kind of information. And I want a solution that works in the broadest extension of cases:

  • I want it to work without SHA-1; I guess this starts already to be difficult; while it’s not clear whether SHA-1 is weak enough to be a vulnerability or not, being able to ignore the issue by using a different algorithm is certainly a desirable feature;
  • I want it to work with GnuPG and OpenSSH at least; if there is a way to get it to work with S/MIME it might also be a good idea;
  • I want it to work on both Linux and Mac OS X: I have two computers in my office: Yamato running Gentoo and Merrimac running OSX; I have to use both, and can’t do without either; I don’t care if I don’t have GnuPG working on OSX, I still need it to work with OpenSSH, since I would like to use it for remote access to my boxes;
  • as an extension to the previous point, I guess it has to be USB; not only I can switch it between the two systems (hopefully!), I’m also going to get a USB switch to use a single console between the two;

I guess the obvious solution would be a tabletop smartcard reader with one or more cards (and I could get my ID card to be a smartcard), but there is one extra point: one day I’m going to have a laptop again, what then? I was thinking about all-in-one tokens, but I have even less knowledge about those than I have about smartcards.

Can anybody suggest me a solution? I understand that the FSFE card only supports 1024 bit for the keys, which seems to be tied to weakness lately, no idea how much of that is true though, to be honest.

So, suggestions, very welcome!

The mess I found with PAM

So I started working, as I anticipated before, toward centralising a bit of PAM configuration for Gentoo in a pambase package. It’s still in flux a bit, but the base should be here already.

The 20080219.1 ebuild now install system-local-login and system-remote-login. ConsoleKit and similar should go to system-local-login, while system-remote-login is reserved to ssh and similar software.

I opened a tracker bug for the new pambase system, I’ve started the cleanup process for pam_passwdqc (which will be available as alternative to cracklib in the pambase ebuild soon, so that it also allows a FreeBSD-like default – FreeBSD uses pam_passwdqc by default), and I started converting ebuilds.

The new setup is quite flexible, I think, but it’s not really complete yet. Unfortunately I’m dedicating just part of my daily time on this, as I’m also working on C# (and swearing against Microsoft for their broken documentation about clipboard usage!). I’m still uncertain on a couple of points, but it should come out nicely in the end.

I’m also wondering if I should move ftpd support from ftpbase into pambase, it makes sense to me as that way it is handled properly by all current and future PAM implementations supported by Gentoo. ftpbase is currently maintainer-needed, so it makes even more sense to bring at least part of it under a team, even if understaffed as PAM team is.

I’ll also have to talk with the net-mail team about mailbase for similar reasons.

Unfortunately, doing this is also showing off big problems on how PAM was handled up to now. There are packages with a pam USE flag, but not installing any pam.d configuration file, there are packages that install upstream-provided files that are not valid in Gentoo, there are packages still providing configuration files referring to pam_stack (in comments). I’m starting to wonder if I should ask the Portage (and QA) team to add an extra QA check for PAM configuration files after build, so that I can find more easily packages breaking the rules.

I admit it’s not easy on the maintainers’ part as I haven’t finished writing the PAM documentation. I hope to be free from work in the next weeks, but I start to wonder if I’ll ever be. And I also hate working on PAM as always, but somebody has to do that, no?

Anyway, two packages are plainly broken and not looking up to be fixed: net-misc/ssh (which should be punted from the tree ASAP, thanks Gustavo!) and net-misc/lsh (another SSH implementation, it has completely broken PAM – it checks against other, which is set to deny any user – and no maintainer; if you want to save it, step up, and I might actually fix the PAM part).