Proper QA does not mean being perfectionists

As a few people already asked me directly which were the so-called «real reasons» for my leaving. As I said, the straw that broke the camel’s back was the one I already expressed in my goodbye post, and it’s not concealing a different reason, for that. Still, there are plenty of reasons that got me tired, and that predisposed me for leaving so quickly when I really grew tired.

I given a brief interview to ABC Linuxu (the article is available in Czech, I do hope it was translated correctly, Jakub seems to think so, so I trust him on this), but of course I cannot point everybody just to that. I’ll then see to dedicate a few blog entries on this topic, now that my cold seems to have wore off and I’m concentrating on getting my job confirmed after this month.

So let’s start with one of my main concerns about the status of Gentoo when I left it: the Quality Assurance idea, and policy being «enacted».

First of all, I want to point out that I’m not targeting Stephen alone on this; while I cannot say I really do like him, I still respect him to an extent, and he was a mentor so I still owe him something. The problem here is more embedded in the whole Gentoo developers pool, as also Mark (who I also respect, sometimes more than Stephen, if that says something) was going on the totally wrong direction, in my opinion.

So, let’s start to see what I mean. All the past Quality Assurance efforts, that fell into the QA project and that were labeled QA, were focused on enforcing a code correctness on ebuilds, eclasses, profiles, practises, processes, and so on. I’m not saying this is not a good idea, but I do think that this is not what the Quality Assurance team should be focusing on; this should be more like a «code police» project and team, limited, which could make sure that of all the possible variants way to do something, the most correct formal one is chosen every time; calling it «code police» can be harsh, but calling this Quality Assurance is not only misleading, but also diverges forces from the actual need for a Quality Assurance team and process.

So, what do I think the QA team should take care of? First of all, it should focus on the Gentoo user side, rather than on the internal development: Quality is an attribute of the final product, not of the internal state of the ebuilds. Nobody will notice, nobody will care, if the ebuilds are totally correct on the code side, but they don’t work well together, and trust me, I know quite a few examples of ebuilds that needed to be better handled in the past, and in the current situation too.

The first example are those ebuilds that are marked stable for an architecture (mostly x86) but does not work at all, not even build sometimes, because newer version of their dependencies dropped something, or one of the old dependencies’ dependency is no more present and the ebuild should have also depended on that one. You don’t think this is so common? I can tell you that CJK team had quite a bit of problems with this kind of ebuilds, especially because of modular Xorg, and ebuilds like jfbterm that were missing dependencies on xorg-x11 or alternatives, as they didn’t use it directly.

One thing that helped a lot on this for CJK team was the work of Patrick Lauer with the tinderbox. The bugs he filed about ebuilds not completing their build phase often involved app-i18n packages, and that allowed me to find some fonts packages that needed to be revbumped, or marked stable, and jfbterm itself, that was totally broken on stable.

Most of the time, the secret of a good Quality Assurance, as experienced by users, is a proper communication between teams; sometimes this needs to either wake up a team, or to take its place, or to find replacement for it. While Donnie and Josh did a hell of a good job to handle the Modular Xorg migration, and we are all happy to see how smooth it ended up, some of the packages needing a new stable after the modular packages gone stable too were missing because, well, the teams weren’t there at all. I joined CJK team afterward, or I would have taken care of that myself, but before that there was just a mess, because a team wasn’t responsive, nor existing.

A similar problem happens when a new package is added that breaks API, rather than ABI. Thanks mainly to Alexis, FLAC bump to 1.1.3 will be pretty much painless for Gentoo users: it was introduced in package.mask by me, and we both checked every package depending on FLAC. Unfortunately not every bump is handled this way because, well, there is a lot of work to do for handling it this way, and leads to higher delays into getting the package available for users to test, increasing in turn the maintenance burden for the package in overall. Many developers prefers handling this by using an overlay, to a similar result, although I do prefer handling everything in main tree when possible, or should I say «did»? I’m no more a developer so I don’t have any voice in this anymore.

There are also other problems, that are already labelled as QA problems, like the libraries missing soname or needed entries (this latter in turn is a source of --as-needed incompatible software, if you wondered why I always interested in that detail), but most of these misses a clear documentation, and they are not given any priority over the correctness of ebuild files for corner-cases (like ROOT != /, that strictly speaking is only needed for a subset of the packages for catalyst).

So my point on this is that the name of QA is being totally misinterpreted by almost the whole developers pool (or at least so appears as very few people spoke against the current situation), and what we’re in front of is a simple case of excessive perfectionism. Quoting Voltaire:

Le mieux est l’ennemi du bien. The best is the enemy of the good.

(I regret not having studied French in more than eight years now…)