*This is a rant. This is not meant to be constructive, so it is not. If Rich is reading this, he might find some points he can work on; if I had the time, I would be working on them myself; if you feel like you agree with my rant and would like to get the stuff implemented, you can hire me to work on this. But for the rest, this is a rant, so if you’re not in the mood to read my rants, you might want to skip over this.*
The laptop from hell is finally shaping up; the smartcard reader works, after editing the ccid files (yes I have to publish the patches up there). Thanks to this I finally wanted to take one further step to make use of it to augment my productivity. One of these things, which I couldn’t feasibly do with OS X, is handling my virtual machines park via virt-manager.
Now, libvirt is designed to be a client-server system, and the server might not be local. There are three transport options to use remote servers: clear-text, TLS, and SSH tunnelling. Now, the clear-text is an obvious bad choice; SSH would be a good choice, if it wasn’t that.. it only works with the root user. There is no way to configure which user to use for the ssh tunnel, and I really don’t want to enable SSH root logins.
TLS is an interesting choice. With this transport, the authentication on the server side is done similarly to what OpenVPN does: you have a personal certification authority (CA), one key/certificate pair for the server, and then one pair per client. All the certificates are signed by the CA itself, and that validates the client to connect to the server. It’s a very nice approach for hosting providers I guess, since you can have a number of workstations that have the certificates set up properly to connect to the farm of virtualisation servers. Unfortunately it has design flawswhen you want to use it with something like a laptop.
First of all, while the server’s certificates and key files’ paths are configurable in the libvirtd.conf file, the client’s files are not configurable, they are hardcoded at build-time based on the system configuration directory (for Gentoo, that’s /etc). They are also only used host-global, as it does not even check for an override in the user’s directory. And this is a double-problem because the certificate has to be passwordless! Or, to put it in a different way, it has to be insecure, lacking a password protection. And since I just said that it does not allow for per-user overrides, you cannot even rely purely on encrypting your home directory. Alternative option is to symlink them from the /etc paths to your home directory but that’s not elegant at all.
It gets even a little worse: to access the display of the virtual machines libvirt tries to do a smart thing, by using the VNC protocol rather than reinventing the wheel. Now, a lot of comments can be written regarding the choice of using the VNC protocol itself, but the fact that it doesn’t reinvent a new one is positive. Unfortunately, when accessing a remote server via TLS, instead of muxing the VNC protocol over the same connection, it simply tries to connect to the VNC display on the remote host. And of course it’s a separate configuration to tell qemu to open the VNC on the correct host. D’oh!
Okay so I get to configure qemu to open the VNC on all interfaces, all IPs… but here’s the catch: for TLS to work correctly, you need to provide correct hostnames, stable hostnames; to do so I decided to use IPv6, since my boxes’ IPv6 are already forward-confirmed, thanks to the latest Hurricane Electric service (providing name server hosting without asking me to maintain my own). Unfortunately, it seems just like qemu does not support IPv6 at all, which means that .. the connection will not work because the hostname will find no hit.
It really doesn’t look too difficult to implement at least part of these features, like choosing the certificate files path, or connecting as a different SSH user. Sure, if you were to shoot high, you could probably consider using a single SCTP socket with multiple channels to multiplex the libvirt protocol together with the VNC connections, but that’s not needed at all. It really just need a few touches here and there to make it much more usable.
I keep hitting problems with libvirt all the time. Looked at alternatives and am planning to poke into ganeti. Did you give it a try?
I just connect to user@host as hostname and have root logins disabled. I might be abusing something, but it works for me. In fact, virt-manager (0.8.4-r2 from the tree) translates this into ‘ssh -l user host nc -U /var/run/libvirt/libvirt-sock’.Another rant: it does leave SSH connections open after I’ve closed virt-manager.
Ssing ssh with non-root users definitely works!I have>>unix_sock_rw_perms = “0770”<<in my libvirtd.conf and set a group on /var/run/libvirt-sock and each user in this group and use virt-manager via ssh to manage VMs …
Hmm which nc program are you guys using? Because here it still fails to work at all..
First of all, make sure the user has read and write permission on the libvirt socket. The easiest way is given by Markus and I believe it’s default on recent versions to use the libvirt group for this. See unix_sock_group in libvirt.conf for your value.nc is net-analyzer/netcat6-1.0-r2 since app-emulation/libvirt-0.8.1-r1 requires >=net-analyzer/netcat6-1.0-r2.Another trick you could use (if you *have* to enable root), is PermitRootLogin without-password in sshd.conf. This disables password logins and can be used to force pubkey auth.
In virt-manager, specify the ssh hostname as “user@vmhost”. Sucks, but it works.
It actually isn’t _that_ easy… * thanks to Ewoud, Markus and Chris, I now know that (without advertising it) SSH tunnels work with non-root, that’s good; * unfortunately, they’ll fail for both root and not if you even have policykit enabled in the build; * there is no sanity check for the QEmu configuration: if you set the VNC to use TLS before, it’ll try to forward that connection but will *not* use TLS on the client, so it’ll not work; * when forwarding the QEmu VNC connection, the command passed to ssh is overquoted; somehow bash can act on it on itself, but zsh (which happens to be my preferred shell, and the one set on my user) tries to execute a single command named “nc -U localhost 5900”, and obviously fail; * as Ewoud said, it leaves stray SSH connections around, even worse it leaves stray SSH _processes_ around, zombified; neither are that good; * sometimes it crashes when re-creating the VNC connection, I’m quite not sure on what happens there.I guess there is still quite the space for improvements.
I guess you filed bugs for all these?Complicating the picture is there are two separate “nc” programs. The one used by Fedora and the one used by Debian have separate lineages and support different command line options. It’s quite possible that libvirt / ssh / nc only works with the Fedora one, although I seem to recall at least trying to fix this in an earlier version of libvirt.
Nope, not yet, I was meant to write you about it, actually. Mostly I hadn’t had time to look up which parts of the code did what; for qemu I was actually hoping to get someone to patch it for IPv6 support…
As others have mentioned, you can use SSH as a non-root user, but you must first configure libvirtd’s UNIX domain socket to be accessible to that user account.Current QEMU already supports IPv6 for VNC. If you use ‘-vnc hostname:1’ it should resolve ‘hostname’ and if it resolves to an IPv6 address use that. You can also tell it explicitly to ignore either IPv4 or IPv6 addrs from DNS if required.