Changes to the netatalk ebuild

You might remember a few weeks ago I bought a new system to use as a local fileserver, keeping among other things a complete copy of the distfiles mirrors (and more than that) to use with the tinderbox and the other boxes. But that system is not only limited to serve (via NFS) the distfiles for my systems, it’s also set up to serve as main storage point for customers’ data as well as my own data, including my music library (which is primarily managed by iTunes, but is consumed by Banshee as well) — before this, a rsync task copied the iTunes library over Yamato to then serve it to yet another box.. you probably see how duplicating this data is not something I’m happy about.

Well, turns out that Samba can’t get me anything over 50Mbit of bandwidth over a gigabit network, when copying over the data, not nice (and this is without considering that each time I copy over files with Windows 7 I have to answer “Try again” as it reports a network-busy error…). So I decided to try again Netatalk, that I used to use a long time ago, and the ebuild of which I have partly wrote myself in the past — the result has been satisfying: 200Mbit, which is still not gigabit, but it’s still four times as fast as Samba.

Now, since my two OS X systems (okay, three with my mother’s) are all running Lion, what I needed was the 2.2 series (2.2.1 release) with Avahi support, not a problem, just need the ~arch version for now. But after using it for a week or so, I started seeing a number of issues that needed to be addressed. The most obvious of which was that restarting Avahi didn’t cause the AFP daemon to restart, which in turn meant that OS X was unable to connect to the fileserver after reboot. D’oh.

The Netatalk ebuild seems to have been maintained on life support for a while, and simply bumped without checking out all of its internals, so I decided to give it a look. The cause of the issue I hit myself was obvious: upstream-provided init script wasn’t designed to be well-integrated into Gentoo. This is common: upstream projects provide some init scripts that do work with Gentoo but don’t follow our guidelines, which would be especially difficult since we do lack good documentation on how to properly write init scripts for the Gentoo init system.

Thankfully, after an afternoon toying with the init scripts and the ebuild, I was able to get a new revision of the ebuild in tree that, while causing a few changes in behaviour, should be much easier to deal with in the future. Since there isn’t an official documentation for Netatalk in Gentoo, it should be a good idea to document here the changes, so that if somebody is confused by the new ebuild, or has comments on what I have done, it can be used as a reference.

The most noticeable difference from the old design is that the new ebuild installs split init scripts for the services, rather than using the single /etc/init.d/netatalk script. This is important for two main reasons: it no longer risk leaving daemons running if they were enabled earlier and stopped later, and it allows to check out the service’s status daemon per daemon.

In the default configuration (the details of which I’ll get to in a moment), netatalk ebuild installs two services: afpd and cnid_metad; the former is the actual file server daemon, the latter is its backend: it provides the CNID metadata for the volumes, and is basically a huge database process. Having the two separate is handy: you no longer need to reload the database when changing the configuration file, which could be a waste of time if you have huge volumes, or volumes with a huge number of files. And most importantly, only afpd talks to the network and needs avahi support, anything else is, well, backend.

The new init scripts don’t rely on the old /etc/netatalk/netatalk.conf configuration file; this happens by design, as the services don’t really share many settings: most of the variables were used to tell which services to start, and the rest were mostly used to pass custom options to the daemons themselves. In the case of afpd almost all the settings passed were configurable in the afpd.conf setting file as well, and the end result is that using that configuration is the suggested method, instead of passing the options via the init script.

The only option that is shared across different services is related to the AppleTalk protocol, which is what I’ll be talking about now. While most users of Netatalk will likely only need the AFP service, which is the file server itself, the package implements also some other services to be used in AppleTalk-based networks. For those who don’t know the details of this, AppleTalk was Apple’s own local network protocol, and was, and sometimes is, used with Ethernet networks with a “wrapper” called EtherTalk.

Apple discontinued support for this technology with the Snow Leopard (10.6) release; and even Leopard itself preferred TCP/IP over AppleTalk, whenever possible, so the uses of that protocol (or to be precise, of DDP) are pretty rare. One of my customers still have a Linux box configured with EtherTalk, because their Mutoh large format printer was configured using that.. but even that’s going to change at some point in the future. With this in mind, there is the other noticeable difference from before.

While netatalk before installed everything for AppleTalk users by default, now there is an appletalk USE flag that needs to be enabled: without that the atalkd service is not installed, nor are the a2boot, timelord and papd services, that are used, respectively, for network boot, time synchronisation and print server hosting (after all, CUPS is now developed by Apple itself). This is another good reason why the services are now split: those three services are only installed if you want AppleTalk support, and they all depend on the atalkd service; afpd on the other hand will use atalkd only if you configure it to do so, which is not the default for most systems.

But I said that AppleTalk users had shared options between services, and that is true: the name and zone for the atalkd service need to be configured; this is now done through /etc/conf.d/atalkd (and you have to follow the same setting with the extra parameters for afpd if you want that to run over AppleTalk). By default it’ll export the host’s short name as the AppleTalk host name, and will use no zone, and that should do the trick for almost every user out there who is still stuck with AppleTalk, for what I can tell.

And to complete the discussion for what concerns AppleTalk, there is one catch: the Netatalk package needs kernel help to support the protocol stack; this means that it should check the kernel configuration and give you hints on what to enable and why. Unfortunately I have no real idea of which settings need to be enabled, so I didn’t add any check for that right now. If you wish to send a patch to do so, it’ll be very welcome.

But not all changes are related to the services: new USE flags were introduced to deal with access control lists (ACLs) and user space quota, and the old XFS USE flag has been removed: it made sense back in the early days of kernel 2.6 as not everybody had the same set of Linux kernel headers in the system, but nowadays, that is only legacy, so support for XFS-style quota is enabled whenever Quotas are enabled as well, just like it was supposed to. Very basic support for LDAP sharing of users and group is also present now, but like for Kerberos, it’ll need a complex testing network to actually work as it was intended, so if you notice anything wrong, please report. And if you only need AFP to work with modern OS X installation, feel free to disable the ssl USE flag, as that one only adds the old, DHX 1 user access module (UAM), which is replaced by the libgcrypt-based DHX2 for modern systems.

The change I’m probably most happy about, though, is the replacement of the statically-linked libatalk library with a shared object, that is used both by the utility and as an exported interface (I’m not sure if it should have been exported, but right now it is, so there..). This allows to cut the size of the package from over 8MB down to 4MB, with debug information included, which means also a smaller memory footprint when you have more than one service started (and you always have at least two, possibly three, at any time).

Unfortunately this one has brought at least one issue, which is now fixed: since static linking is non-transitive but also non-asneeded-influenced, external library dependencies were not expressed on the common library, but rather on the tools themselves; this was a problem now, so for a little while the tcpd USE flag couldn’t produce a final link… I fixed that one, but there might be other issues with other combination of USE flags, but I guess this is why we call it “testing”…

Anyhow… if you have suggestions to provide or bugs to report, please don’t refrain from leaving a comment here or opening a bug in our bugzilla — for a while at least, I’ll be the netatalk dedicated maintainer.. again… Next week’s tasks include sending the changes upstream so that we won’t need to keep patching this forever.