I have written before about some of our soft spots for what concerns Gentoo’s quality. Today I’m writing a quite inflammatory piece that most likely will cause me to take a bit of flak, but which I’m pretty sure it’s going to the point.
The main weakness in Gentoo’s Quality Assurance is developers not giving a shit about quality.
Let me try to explain: there are many problems with Gentoo’s QA team, like the lack of proper coordination most of the time, the lack of a real Gentoo-based effort for continuous integration testing (the tinderboxes are mostly direct pet projects of the people running them, which means for the QA standpoint Patrick and me), and the lack of an absolute rulebook of “dos ad don’ts” for what concerns ebuilds.
Of the few accepted rules, there are a few that I’m trying to enforce via the tinderbox and by opening bugs, one of these is installing documentation in the /usr/share/doc/${PF} path (where ${PF} is the full name and version of the package, including the revision number — yes I know that it would have been better to use ${CATEGORY}/${PF} but let’s not go there for now). To do so you either use the Portage-provided dodoc and dohtml helpers, or you have to tell the configure script or the install makefiles to change the directory they install their documentation into; the defaults are never right because at the worst case they don’t take the version into consideration and at the best have no idea about the Gentoo-specific revision.
It doesn’t seem like a controversial rule to enforce, does it?
Well it seems like some developer (whom I’m not going to out just yet) doesn’t like that idea, and even if it’s now very well known that the bugs I’m opening from the tinderbox are vastly on behalf of QA decided to close two of my bugs as WONTFIX. And yes that pisses the hell out of me. Even more because the packages installing in the wrong directory do so because their upstream-provided source packages has been voluntarily split to provide a temporary counter-measure to the lack of proper USE dependencies (and this has caused more than a little grief I’d say).
But it’s not the first time I get responses on this line; for instance some developers find too bothersome to explicitly list the upstream-provided precompiled (sometimes proprietary sometimes not) files that fail important QA/security checks like RPATHs and TEXTRELs, and thus prefer simply restricting binary checks entirely (note: binchecks also fixes vulnerable and wrong RPATHs such as empty entries or current-work-directory).
We got all the “let’s take shortcuts” developers who prefer to commit half-thought stuff without masking it or without experimenting with it for a while to flesh out eventual requirements. While importing a huge amount of code that was developed outside of tree is always a risk (as the Ruby-NG effort is showing pretty well), simply pushing eclasses that support new versions of a language without allowing the user to disable them or without checking for proper reverse dependencies is a pretty bad idea.
Further on the problem comes with developers that drop patches without checking whether they are needed, and without warning that the problem might re-surface (for instance I know that I half-blindly dropped some patches from Linux-PAM; on the other hand I’m currently working on making it work on uClibc again).
So in general, yes, the problem with Gentoo’s QA is mostly developers, and yes, I know wee should come up with some rulebook that is “there or nothing”, I’ll see what I can do about that with the time I got.