The overengineering of ALSA userland

This is a bit of an interesting corner case of a rant. I have not written this when I came up with it, because I came up with it many years ago when I actively worked on multimedia software, but I have only given it in person to a few people before, because at the time it would have gained too much unwanted attention by random people, the same kind of people who might have threatened me for removing XMMS out of Gentoo so many years ago. I have, though, spoken about this with at least one of the people working on PulseAudio at the time, and I have repeated this at the office a few times, while the topic came up.

For context you may want to read this rant from almost ten years ago by Mike Melanson, who was at the time working for Adobe on Flash Player for Linux. ~~It’s a bit unfortunate that the drawings from the post are missing (but maybe Mike has a copy?)~~ You can find the missing drawing on the Internet Archive as well, but the whole gist is that the Linux Audio API were already bloody confusing at the time, and this was before PulseAudio came along to stay. So where are we right now?

Well, the good news is that for the most part things got simpler: aRTs and ESounD are now completely gone, eradicated in the favour of PulseAudio, which is essentially the only currently used consumer sound daemon. Jack2 is still the standard for the pro-audio crowd, but even those people seem to have accepted that multimedia players are unlikely to care for it, and it should be limited to proaudio software. On the kernel driver side, the actually fairly important out-of-kernel drivers are effectively gone, in favour of development happening as a separate branch of the Linux kernel itself (GIT was not a thing at the time, oh how things have changed!) and OSS is effectively gone. I don’t even know if it’s available in the kernel, but the OSS4 fanboys have been quiet for long enough that I assume they gave up too.

ALSA itself hasn’t really changed much in all this time, either in the kernel or as userland. In the kernel, it got more complex for supporting things like jack sense, as HDA started supporting soft-switching between speaker and headphones output. In the userland, the plugins interface that was barely known before is now a requirement to properly use PulseAudio, both in Gentoo and in most other distributions. Which effectively makes my rant not only still relevant, but possibly more relevant. But before I go into details, I should take a step back and explain what the whole thing with userland and drivers is, with ALSA. I’ll try to simplify the history and the details, so if you know this very well you may notice I may skip some details, but nobody really cares that much about those.

The ALSA project was born back when Linux was in version 2.4 — and unlike today, that version was the version for a long time. Indeed up until version 3.0, a “minor” version would just be around forever; the migration from 2.4 to 2.6 was a massive amount of work and took distributions, developers and users alike a lot of coordination. In Linux 2.4, the audio drivers were based off the OSS interface, which essentially meant you had /dev/dspX and /dev/mixerX, and you were done — most of the time mixer0 matched a number of dspX devices, and most devices would have input and output capabilities, but that’s about all you knew. Access to the device was almost always exclusive to one process, except if the soundcard had multiple hardware mixer channels, in which case you could open the device multiple times. If you needed processes to share the device, your only option was to use a daemon such as the already named aRTs or ESounD. The ALSA project aimed to replace the OSS interface (that by then became a piece of proprietary software in its newer versions) with a new, improved interface in the following “minor” version (2.5, which stabilized as 2.6), as well as on the old one through additional kernel modules — the major drawback from my point of view, is that this new interface became Linux-specific, while OSS has been (and is) supported by most of the BSDs as well. But, sometimes you have to do this anyway.

The ALSA approach provides a much more complex device API, but mostly for good reason, because sound cards are (or were) a complex interface, and are not consistent among themselves at all. To make things simpler to application developers who previously only had to use open() and similar functions, ALSA provided an userland library, provided in a package called alsa-lib, but more often known as its filename: libasound. While the interface of the library is not simple either, it does provide a bit of wrapping around the otherwise very low-level APIs. It also abstracts some of the problems away of figuring out which cards are present and which mixer refers to which. The project also provided a number of tools and utilities to configure the devices, query for information or playback raw sound — and even a wrapper for applications implementing OSS access only, in the form of a preloadable library catching accesses to /dev/dsp to convert them to ALSA API calls — not different from the similar utilities provided by arts, esd or PulseAudio.

In the original ALSA model, access to the device was still limited to one process per channel, but as soundcards with more than one hardware channel became quickly obsolete (particularly as soundcard kind-of standardized over AC’97, then HDA) the need for sharing access arose again, and since both arts and esd had their limits (and PulseAudio was far from ready), the dmix interface arrived — in this setup, the first process opening the device would actually have access, as well as set up a shared memory area for other processes to provide their audio, which then would be mixed together in userland, particularly in the process space of the first process opening the device. This had all sorts of problems, particularly when sharing across users, or when sharing with processes that only used sound for a limited amount of time.

What dmix actually used was the ability of ALSA to provide “virtual” devices, which can be configured for alsa-lib to see. Another feature that got more spotlight thanks to the lowering of featureset in soundcards, particularly with the HDA standard, is the ability to provide plugins to extend the functionality of alsa-lib — for a while the most important one was clearly the libsamplerate-based resampling plugin which almost ten years ago was the only way to provide non-crackling sound out of an HDA soundcard. These plugins included other features, such as a plugin providing a virtual device for encoding to Dolby AC3 so that you could us S/PDIF pass-through to a surround decoder. Nowadays, the really important plugin is the one PulseAudio one, which allows any ALSA-compatible application to talk to PulseAudio, by configuring a default virtual device.

Okay now that the history lesson is complete, let me see to write down what I think is a problem with our current, modern setup. I’ll exclude in my discussion proaudio workstations, as these have clearly different requirements from “mainstream” and most likely would still argue (from a different point) that the current setup is overengineered. I’ll also exclude most embedded devices, including Android, since I don’t think PA ever won over the phone manufacturers outside of Nokia — although I would expect that a lot of them actually do rely on PulseAudio a bit and so the discussion would apply.

In a current Linux desktop, your multimedia applications end up falling into two main categories: those that implement PulseAudio support and those that implement ALSA support. They may use some wrapper library such as SDL, but at the end of the day, these are the two APIs that allow you to output sound on modern Linux. A few rare cases of (proprietary, probably) apps implementing OSS can be ignored, as they would either then use aoss or padsp to preload the right library to provide support to whichever stack you prefer. Whichever distribution you’re using all of these two classes of apps are extremely likely to be going out of your speaker through PulseAudio. If the app only support ALSA, the distribution is likely providing a configuration file so that the default ALSA device is a virtual device pointing at the PulseAudio plugin.

When the app talks to PulseAudio directly, it’ll use its API through the client library, that then IPCs through its custom protocol to the PulseAudio Daemon, which will then use alsa-lib through its API, ignoring all the virtual devices configured, which in turn will talk with the kernel drivers through its device files. It’s a bit different for Bluetooth devices, but you get the gist. This at first sight should sound just fine.

If you look at an app that only supports ALSA interfaces, it’ll use the alsa-lib API to talk to the default device, which uses the PulseAudio client library to IPC to the PulseAudio daemon, and so as above. In this case you have alsa-lib on both sides: the source application and the sink daemon. So what am I complaining about? Well here is the thing: the parts of ALSA that the media application uses versus the parts of ALSA that the PulseAudio daemon uses are almost entirely distinct: one only provides access to the virtual devices configured, and the other only gives access to the raw hardware. The fact that they share the API is barely a matter, in my opinion.

From my point of view, what would be a better solution would be for libasound to be provided by PulseAudio directly, implements a subset of ALSA API, that either show the devices as the sinks configured in PulseAudio or, PA wants to maintain the stream/sink abstraction itself, just a single device that is PulseAudio. No configuration files, no virtual devices, no plugins whatsoever, but if the application is supporting ALSA, it gets automatically promoted to PulseAudio. Then on the daemon side, PulseAudio can either fork alsa-lib, or have alsa-lib provide a simpler library, that only provides access to the hardware devices, and removes support for configuration files and plugins (after all PulseAudio already has its own module system.) Last I heard, there actually is an embedded version of libasound that implements only the minimal amount of features needed to access a sound device through ALSA. This not only should reduce the amount of “code at play” (pardon the pun), but also reduce the chance that you can misconfigure ALSA to do the wrong thing.

Misconfiguring ALSA is probably the most common reason for your sound not working the way you expect on Linux — the configuration files and options, defaults and so on kept changing, and since ten years ago things are so different that you’re likely to find very bad, old advise out there. And it’s not always clear not to follow it. And for instance for the longest time Adobe Flash, thinking of doing the right thing, would not actually abide to the default ALSA configuration, and rather try to access the hardware device itself (mostly because of nasty bugs with dmix), which meant that PulseAudio wouldn’t be able to access it anymore itself. The quickly sketched architecture above would solve that problem, as the application would not actually be able to tell the difference between the hardware device and the PulseAudio virtual device — the former would just not be there!

And just to close up my ALSA rant, I would like to remember you all, that alsa-lib still comes with its own LISP interpreter: the ALISP dialect was meant to provide even more configurability to the sound access interface, and most distributions, as far as I know, still have it enabled. Gentoo provides a (default-off) alisp USE flag, so you’re at least spared that part in most cases.

Update 2020-05-08: Adobe appears to have “archived” (or rather deleted) Mike’s blog from their site, some time in 2017. I’ve updated the link above to point at the Internet Archive (to which I recommend donating, if you can). While doing that I even managed to find a copy of the drawing that was missing when I originally wrote this post. Score!

Revisiting my opinion of GitHub — How do you branch a readonly repository?

I have expressed before quite a bit of discontent of GitHub before, regarding the way they continue suggesting people to “fork” projects. I’d like for once to state that while I find the fork word the wrong one the idea is not too far from ideal.

Fast, reliable and free distributed SCMs have defined a new landscape in the world of Free Software, as many probably noted already; but without the help of GitHub, Gitorious and BitBucket I wouldn’t expect them to have made such an impact. Why? Because hosting a repository is generally not an easy task for non-sysadmins, and finding where the original code is supposed to be is also not that easy, a lot of times.

Of course, you cannot “rely on the cloud” to be the sole host for your content, especially if you’re a complex project like Gentoo, but it doesn’t hurt to be able to tell users “You want to make changes? Simply branch it here and we’ll take care of the management”, which is what those services enabled.

Why am I writing about this now? Well, it happened more than once before that I was in need to publish a branch of a package whose original repository was hosted on SourceForge, or the GNOME source , where you cannot branch it to make any change. To solve that problem I set up a system on my server to clone and mirror the repositories over to Gitorious; the automirror user is now tracking twelve repositories and copying them over to gitorious every six hours.

As it happens, last night I was hacking a bit at ALSA — mostly I was looking into applying what I wrote yesterday regarding hidden symbols on the ALSA plugins, as my script found duplicated symbols between the two Pulse plugins (one is the actual sound plugin the other is the control one), but ended up doing general fixing of their build system as it was slightly broken. I sent the patches to the mailing lists, but I wanted to have them available as a branch as well.

Well, now you got most of the ALSA project available on Gitorious for all your branching and editing needs. Isn’t Git so lovely?

Before finishing though I’d like to point out that there is one thing I’m not going to change my opinion on: the idea of “forking” projects is, in my opinion, very bad, as I wrote on the article at the top of the page. I actually like Gitorious’s “clones” better, as that’s what it should be: clones with branches, not forks. Unfortunately as I wrote some other time, Gitorious is not that great when it comes to licensing constraints, so for some projects of mine I’ve been using GitHub instead. Both the services are, in my opinion, equally valuable for the Free Software community, and not only.

Update (2017-04-22): as you may know, Gitorious was acquired by GitLab in 2015 and turned down the service. This means that all the references are totally useless now. Sorry.

Gentoo/PulseAudio Summer 2009 Plans

Against to avoid the problem of bus factor, I’m going to write down here what the plans are, for what concerns me, with PulseAudio and Gentoo for this end of Summer 2009, mostly related to what will happen when I’ll come back from my vacations in London, after mid August.

This actually is also out of candrews asking for it as I haven’t really thought about writing this before that.

So the first thing to say is that I am following PulseAudio pretty well; or rather I’m following Lennart pretty well (he’s also the one that suggested me to rewrite udev’s build system to use non-recursive automake — something I’ll write more about another day), so I’m not sleeping waiting.

Indeed, the 0.9.16 test releases are available in Gentoo already, although masked, and since recently they both support udev hotplug (preferring it over HAL), and also pass all the checks already. A note on the tests is needed though: the mix-test lacks a few entries, in particular regarding 24-in-32-bit samples, and is for this reason disabled in the current ebuild (Lennart should be working on it); at the same time, the ebuild is running test specifically in the source directory, because the intltool checks fail; badly. In theory the problem should be fixed in 0.41 series of intltool, but I am unsure whether that should be packaged or not by us.

In the next release, whether it’ll be another test release or the final release, there will also be a few differences in the handling of audio APIs. The OSS support will be restricted, masking the USE flag on Linux (leaving it enabled for FreeBSD obviously); this means that users wanting to use stuff like OSS4, which is not in Portage and if it’s for me will never be, will have to go a slightly longer way to get it to work with PulseAudio. The reason for this is that Lennart really don’t want to support that, and I can agree with him. Now, if you know the package well, you’ll probably be wondering “what about the OSS-compatibility wrapper?” this is solved already: in GIT the OSS output and wrapper supports are split in two different options, the former will be tied to the oss USE flag, the latter will be left in “auto-mode”, which will create the padsp rapper on all Gentoo Linux and FreeBSD systems. And this should fix your problem Luca!

As for some of the new features, like for instance Rygel UPnP support, well, I’ll probably be working on the sometime in the future; I do want to get Rygel in portage, especially if that will allow me to look at my vacation’s photos directly on my Sony Bravia networked TV.

PulseAudio and quirks

Seems like even my previous post about PulseAudio got one of the PA-bashers to think I’m a nuisance for their “cause”, whatever that is. For this reason I’d like to try to explain some of the quirks regarding PulseAudio, distributions, quirks and so on. Let’s call this a bit of a backstage analysis of what’s going on about Linux and audio, from somebody that has little vested interested in trying to roll the thing for PulseAudio.

The first problem to address relates to the comments that KDE people find PulseAudio a problem; I guess this has to be decomposed in a series of multiple problems: Lennart is a GTK/GNOME guy, so he obviously provided the original tools for GTK/GNOME. For a while I was interested in writing the equivalents for KDE (3) but I never had the time; now that I also moved to GNOME independently, I sincerely have no intention to write KDE tools for PA… but one has to wonder why nobody in KDE went out of his/her way to try doing this before. It’s not like it had to be part of KDE proper, it would have been okay to be an unofficial standalone application.

There is also another problem: most of the KDE guys who do see problems with PulseAudio are most likely using Phonon with xine-lib backend, configured to use the PulseAudio output plugin. Given I’m the one I wrote most of it originally, I can say that it sucks big time. Unfortunately I have had no time to work on that lately, I hope I might have that time in the future, but the two years I spent between hospitals seriously indebted me to the point I’m doing about 18 hours of work a day on average. For those who do want to use xine-lib with Pulse, I’d like to suggest the long route: set up the ALSA Pulse plugin, and then let xine just use ALSA.

There is of course another problem for KDE: while GNOME historically had no problem with force in dependencies that are Linux-specific or that work most of the time just on Linux (think about HAL adoption for instance), and relied on the actual vendors to do the eventual porting, KDE strives to work most of the time on multiple operating systems, including as of KDE 4 also Mac OS X and Windows. Now you might like this or not, but it’s their choice; and the problem is that while there is some kind of PulseAudio support for Windows, at least OSX is pretty badly shaped (also on my radar).

For what concerns distribution support, it is true that Lennart usually just care about Fedora; you have to accept this as part of the deal given RedHat is – as far as I know at least, Lennart feel free to correct me if I’m wrong – the one vendor paying his bills. Now of course we’d all love to support all the distributions at the same time, but the only way that’s possible is if multiple maintainers do coordinate; I’ve been doing my best to pass all the patches upstream when I’ve added them to Gentoo, and I see Colin Guthrie from Mandriva doing the same. One thing I can “blame” Lennart for (and I told this to him before, too!) is not creating a GIT branch with the cherry-picked patches he applies on the Fedora packaging for us to pick up… and the fact that he doesn’t like neither making releases or leaving access to others to do so.

To be honest, there is little different in this from what other projects do with distributions like Ubuntu when they are paid by Canonical. I think this is obvious, everybody looks at their little garden first. But this is not something that should concern us I guess. Gentoo has been quite out of the loop for what concerns PulseAudio, and I’m sorry, that was mostly my fault. I’m doing my best to let us update as soon as possible, but it’s not just that simple, as I already explained .

Then let me just say something about Lennart’s refusal to support system mode (which is available and advertised in Gentoo since PulseAudio entered the tree): I can’t blame him for that. First, his design for PulseAudio is based on providing something that works for the desktop use case. Something along the lines of Windows’s or OSX’s audio subsystems, neither of which provide anything akin to system mode. And indeed PulseAudio, by design, can handle the same situations, including multi-user setups with fast user switching. The fact that a system mode exists at all is due to the fact that I for one needed something like it on my setup, hacked it around for Gentoo, and then Lennart made my life easier implementing some extra bits on PulseAudio proper, but it was certainly not his idea.

What people complain about usually is the need for an X session (not strictly true, PulseAudio will start just fine in SSH — it would probably be possible to even fix it up so that it would tunnel audio just like you can tunnel X!), and the fact that audio does not continue to work when X exits (also not strictly true, if your audio player is running in screen it would be working just fine; it’s the fact that the media player crashes that makes your audio stop). Additionally people complain about the security problem of wanting to have all the processes to run under the same user, rather than allowing them to be on different users, like mpd.

Well, some complains are valid, other are not: it is true that PulseAudio does not work in multi-seat-multi-user environments, at least not with a single audio device, it is unfortunate and I don’t know if it’ll ever do work in that situation without a system mode. It is also true that running processes as different users for privileges separation does not work without system mode. But both these options are walking quite away from the the desktop design that PulseAudio is implementing; sure they are valid use cases, just like embedded systems (Palm Pre uses PulseAudio if you didn’t notice that before), but they are not what Lennart is interested in himself; at the same time I don’t think he’d be stopping anyone to improve the system mode support for those, as long as it wouldn’t require the desktop setup to make compromises.

Because the idea is, as usual in any software design, the one that you have to take compromises; Lennart wants the best experience for what concern desktop systems, and he compromises that system mode is not part of his plan, and it shouldn’t be hindering him. At the same time, while he does get upset when people ask for support about it, and he wrote why it’s not supported he hasn’t removed it (yet — if I was him, at this point I could have just removed it out of spite!). So colouring him as the master of evil does not seem the very best idea — and especially that makes me picture him in the part of Warren in the Trio, from Buffy’s season six.

Oh and a final note: it doesn’t have to surprise that Lennart and Fedora don’t care about running mpd and other services as different users, there are probably quite a few reasons for this. I cannot speak for Fedora, given I’m not involved in it, but my suppositions are that firstly the ALSA dmix plugin is somewhat scary from a security point of view (for me too) because it uses shared memory between processes from different users to do the mixing, and the second is that Fedora does a lot to use SElinux even on standard desktops. This is much tighter than separating privileges with different users since it forces the processes to behave as instructed. Unfortunately on Gentoo the SElinux support seems to have gone for good, at least to me.

Planning for PulseAudio

Thanks to Betelgeuse I finally have audio again on Yamato (again, thanks! — on a different note, this actually made me find out that there absolutely is a bug in ALSA that causes mmap to kill PulseAudio both with the ICE1712 and the HDA drivers), so I’m resuming my duty as PulseAudio maintainer. This is the reason why PulseAudio jumped to version 0.9.15-r50 in ~arch. So what’s up with that?

My current plans in respect to PulseAudio are trying to get 0.9.15 in stable to replace the ancient 0.9.9. What has stopped PulseAudio to go stable up to this point has been exactly two dependencies: OpenRC and libtool 2.2. Originally, the idea was to keep PulseAudio only compatible with OpenRC and no longer with baselayout 1; it was supposed to go stable pretty soon and the baselayout 1 init script was so scarily incomplete that we simply preferred not have to support it.

Unfortunately, there is still no date for OpenRC to go stable, if it’ll go at all in its current form. At the same time, Lennart has seriously warned against system wide mode (even though there are still valid use cases for which Gentoo often is used!) so keeping the new versions off from stable for a “minor” feature that is not even recommended to be used sounds like a bad plan.

For this reason I’ve now split the ebuild in two versions: one will keep the system mode support, with the system mode warnings, the init script and all the niceties, and the other won’t, and won’t depend on OpenRC at all; the latter is what is supposed to go stable and what stable users should locally unmask if they want PulseAudio.

Let me state again: if you want newer PulseAudio and you’re in stable explicitly request the -r1 version, not the -r50!.

Unfortunately while I should be able to ask for stable right away for what concerns time and bugs, there are a few dependencies, which include libtool 2.2 which is not stable yet (and I think it should be, the tinderbox haven’t found many libtool 2.2 bugs lately and quite a few packages started requiring that, rather than just a generic libtool that 1.5 is compatible with).

I still have no real plans for the realtime support; while Lennart released rtkit (does anybody find it concerning that Linux started having packages with names vaguely similar to those from Apple’s OS X?), it needs a patched kernel, which means I should probably be pestering our kernel team to get those patches included before we can actually provide it, even optionally.

This week I hope to be able to work on mpd too, so that the Gentoo packaging plays nice with PulseAudio (right now the fact that you have to run it with a different user forces you to use a systemwide instance).

Why would anybody need PulseAudio?

That’s a very common question as of lately, and somehow I feel like most people who haven’t dealt with ALSA in the past would find it very difficult to properly answer to it. Even myself I would have ignored one particular issue till last night, when I hit another reason why I want to keep PulseAudio as my main and only audio system as soon as possible, reducing direct ALSA access.

With modern systems you mostly get onboard sound cards, rather than PCI cards and similar, unless you’re somewhat crazy and want a proaudio card (like I did) or a Creative card (not yet sure what’s the fuzz about that). To be honest, my motherboard really does have a limited soundcard so I would have needed a PCI card anyway to get digital S/PDIF output, and I want S/PDIF because the room has too much electric noise to the point I can hear it on the speakers.

Somehow, ALSA seem to hide to most users that there are indeed a lot of limitations with the HDA hardware. This because the idea for the HDA was probably to shift as much work as possible into software space rather than doing so in hardware; this is why you need a driver fix for the headphones jack to work properly most of the time. While this can be seen as a way to produce cheap cards, it has to be said that software always had to compensate for hardware defects since it’s more flexible. And having most of the processing done in software means that you can actually fix it if there is a bug, rather than having to find a workaround if anything.

So, no hardware mixing, and ALSA has to use dmix, no hardware volume handling, and ALSA has to use softvol, no hardware resampling, and ALSA has its own resamplers, and so on so forth. But while ALSA has to cope with doing these things in the same process that are doing the audio output, and thus is not so good at coping with different processes accessing the audio device with different parameters.

Having a separate process doing the elaboration does not mean that you add more work to the system, you can actually make it do less work if, for instance, you ask that process to play a (cached) sound rather than having it opened, converted, and played back.

At the same time, the fact that PulseAudio handles mixing, volume and resampling in software does not mean that it adds to the work done by the software for most cards, since the most common cards nowadays, the HDA-based ones, already do all that in software, just you don’t see that explicitly because ALSA do them in process.

In my opinion, ALSA is really doing way too much as a library, since each time you open a device it has to parse a number of configuration files, to identify which definition to use (and even then, by default they are far from perfect, for instance for the ICE1712-based cards, which depending on the way they are wired may have two, four, six or eight channel outputs, the definition only suits the 8-channel model, and does not make sense with the lower end cards). And once it found the definition it has to initialise a number of plugins, internal or external, to perform the software functions.

And this is all without putting in the mix the LISP interpreter that alsa-lib ships with (I’d leave to Lennart to explain that one).

So if you think that PulseAudio is an over-engineered piece of software that performs functions in software that should be done in hardware, you better be a FreeBSD user. At least there the OSS subsystem does not try to do all the things that ALSA does (and on the other hand FreeBSD is trying to go the Microsoft way for the HDA cards, implementing the UAA specifications rather than having a huge table of quirks; as far as I can see that would be quite useful, if it’s going to work that is).

But even in that case you should probably find PulseAudio a good thing since it tries not to be Linux-specific and thus would allow for performing all the needed functions with a single cross-platform software without having to reimplement it in platform-specific systems like OSS or ALSA. That was my main reason to look into PulseAudio when I was working on Gentoo/FreeBSD.

For everyone who cares about users, let’s try to work all together to make PulseAudio better rather than attacking Lennart for trying to do the right thing, okay?

I’m in yur ALSA… killing your dirties

Well, maybe not right now.

Since I doubt you’d be able to understand what I mean from the lolcat-speech title, let me try to summarise it in a language that nears a lot more what people actually speak (yes I know it’s still going to be too technical for some of the readers, but I guess that cannot really be helped).

Last night I couldn’t sleep, for a series of reason, not last that to make sure I could implement some stuff for my job while waiting for the actual definitive specs, I took three coffee cups, which while making me feel very nice, stops me from sleeping; not so nice when your neighbours woke you up two days in a row fighting, but I can manage.

Since at the time I was waiting for the chroot to complete some builds so I could check and submit a few more bugs (the count of “My Bugs” search on bugzilla now reaches 1200 bugs and has some reserve too), I decided to try something different. I already have been adding to my git repositories changes to a few libraries I contributed to in the past enough buildsystem so that --no-undefined is added, so last night I decided to go with doing some work on ALSA upstream repositories.

I already had checked out three of the repositories when 1.0.18 was added to the tree, since I had to fix an --as-needed issue and decided to just go on and submit all the patches to upstream for merge, this time I checked out alsa-lib, added --no-undefined and then started some analysis with ruby-elf tools cowstats and missingstatic, as well as removed a few compiler warnings, just to make sure I wouldn’t be distracted by faux problems.

The result should now be that the alsa-lib and alsa-plugins libraries have a few dirty pages less, and that the code is a bit more solid than before, with added static and const modifiers where needed. It wasn’t much of a work, but I forgot once again to add -s to the git commits so I had to rewrite history to get the Signed-off-by header to all the commits; if somebody knows how to set git per-repository to always use -s when committing, I’d be very glad.

On the other hand, this task shown me that cowstats still had and has some problems, in particular, it lacked a way to separate .data.rel from .data.rel.ro sections data. This is important to distinguish between the two since .data.rel.ro is fully prelinkable, which means after a prelink it would always be loaded from the disk without further relocation, while .data could still cause copy on write because it can be changed at runtime even after relocation.

This is even further understood by noticing that shared objects built with GCC under Linux have .data, .data and .data.rel.ro, but no .data.rel which is instead merged back into .data itself. But because of this the “real” data count in cowstats is entirely out of reality. I’ll have to rewrite that part most likely.

Anyway, I’ve done my best and hopefully tomorrow more of my patches will be merged in, so that alsa-lib’s dirty pages get reduced again. Unfortunately even after my changes, with all the plugins enabled, and in the worst case scenario, libasound.so will go on requiring more than 28KiB of dirty pages per process (minus forks and various preloads). Which is not nice at all. Prelinking can reduce the dirty pages removing these 28KiB (which are all of .data.rel.ro), and then it would just require a couple of pages.

There is one question though that now is driving me nuts: hasn’t Nokia worked on ALSA for their N800 tablets? I know alsa-plugins has a Maemo plugin (which I also cleaned up a bit last night, as it had quite a few hacks on the autotools side, and an unwarranted use of pow() instead of using left shift), but I’d expect Nokia to know better about having so many dirty pages…

Anyway, for all the remaining users, I strongly suggest you look into removing some of the plugins that ship with ALSA, like the iec958 plugin if you don’t use digital pass-through. By cutting down the amount of built-in plugins you should be able to reduce sensibly the memory that alsa and the applications using alsa would be using on your system.

-I also wonder why didn’t I add an USE flag to disable the alisp feature- Sorry, of course I wouldn’t be able to find an alisp USE flag if I check the output of emerge -pv alsa-utils. D’oh!. Why does ALSA need a LISP interpreter anyway?

Outdated tools

There is one interesting differnce between Linux and full operating system projects like FreeBSD, the other BSDs and OpenSolaris: Linux historically didn’t have much coordination between kernel and userland.

This becomes a problem for instance when udev and the kernel disagree on how to handle something, or on when you end up with a tool trying to use some old kernel interface.

It looks tremendously bad when you see that even strace fail (which on FreeBSD does not seem to happen, as ktrace is part of the single project). And I don’t know of any strace replacement.. and I relay a lot on that tool!

This gets further interesting when you add in the USB access through /proc (usbfs), that has been deprecated a long time ago, but that most people are probably still enabling in their kernel. The new interface, using /dev is available for quite a while, and libusb is supporting it very well. But as it turns out, VMware does not use libusb for accessing the USB devices, and it does not suppor the new interface.

I wonder how many projects have this problem. I remember net-tools being worked on, iproute2 replacing ifconfig and so on… but how many tools are actually always in sync between kernel and userland, as of now?

ALSA is also a very common problem with this as the drivers in the past often ended up out of sync between kernel and driver, causing subtle and obnoxious problems.

And even counting software tools that are well in sync between the two, how many of these tools are being audited for, for instance, performance improvements? I wonder.

I’m afraid this is a blog post without solution, but I’d like to make people think about this, maybe someone can help finding solutions ;)

Blast from the recent past; two years ago

Today I was feeling somewhat blue, mostly because I’m demotivated to do most stuff, and I wanted to see what it was like to work in Gentoo two years ago.

One thing I read is that a little shy of exactly two years ago, ICQ broke Kopete, just like they did yesterday. Interestingly enough, even though a workaround has been found for Kopete 0.12 (the one shipped with KDE 3.5), there is no bump I see in the tree this time. Sign that the KDE support in Gentoo has changed, most likely.

There is also the whole thing with ALSA problems, which span so many posts that it’s not worth listing all them. The current ALSA maintainer simply gave up on providing something that, at least for some users, ended up being quite important.

And all the work on Gentoo/FreeBSD! Although Javier is doing a huge work now to support FreeBSD 7.0, he’s not prone to blog about it, and you can see that Gentoo/FreeBSD is easily ending up in the “historical” memory, rather than being discussed and tried out by users daily.

What didn’t change at all is my insomnia, it’s almost 2AM and I’m still up. And this time I don’t even have Antiques Roadshow to watch. I’m currently working on xine, just like two years ago.

In general, I think a lot of areas in Gentoo did go downhill from two years ago, rather than improving. While Portage is certainly improved, thanks to Zac, Genone and the rest of the team, and we can see that in the new extended repoman checks, that also helps QA. But the general user support seems, to me, lacking.

This is a direct consequence, in my opinion, of leaving open doors for people who are just driving Gentoo’s energy away, by taking over projects to make them stall, by discussing details over and over and over, by repeating the same request even when people reject it as it stands, and so on.

I hope things will improve in the next months, thanks to a new council that can finally grow some balls, straightening up the situation, but if this does not happen, I’m already preparing for my plan B…

Google Summer of Code deadline extension

As you can see on Multimedia Mike’s blog the deadline for Google Summer of Code has been extended to April 7th.

I’ve updated the Gentoo Calendar at Google for the new deadline, too.

I sincerely count as one failure that FreeBSD had an official news item and Gentoo didn’t. We really should learn from these things so I’m pointing it out in public.

I wish xine could have gone into SoC too, but I think there are a few things that has to be cleared out before that could ever happen. I’ll see if I can get myself to that in the next year or so.

On other notes, SoC this year seems to have some interesting projects, and I didn’t see any Gentoo dev submitting applications to Gentoo up to now, which makes me quite happy :) I hope the ones needing the money applied to other projects though.

Now I have to go back to my cabling, but expect me to return to full-fledged status in the next week or so. I’ve been bumping PulseAudio yesterday, and Lennart helped me to find the reason why the M-Audio Audiophile 2496 soundcard is rejected by Pulse, I’ll try to document a bit how to use that with PulseAudio and other audio software as it seems to me like it’s not an easy task, and ALSA is not by default good with it.

By the way, anybody tried Timeshift on PlayStation 3? I tried the demo and the graphic looks seriously cool, but the joypad is not exactly what I’d consider the best input for a FPS. I’ve heard that Unreal Tournament 3 can use the keyboard as a controller, so that makes it more likely for me to buy, but I would have liked to know about other games so that I can choose the most interesting one :)