I have written before about some of our soft spots for what concerns Gentoo’s quality. Today I’m writing a quite inflammatory piece that most likely will cause me to take a bit of flak, but which I’m pretty sure it’s going to the point.
The main weakness in Gentoo’s Quality Assurance is developers not giving a shit about quality.
Let me try to explain: there are many problems with Gentoo’s QA team, like the lack of proper coordination most of the time, the lack of a real Gentoo-based effort for continuous integration testing (the tinderboxes are mostly direct pet projects of the people running them, which means for the QA standpoint Patrick and me), and the lack of an absolute rulebook of “dos ad don’ts” for what concerns ebuilds.
Of the few accepted rules, there are a few that I’m trying to enforce via the tinderbox and by opening bugs, one of these is installing documentation in the /usr/share/doc/${PF} path (where ${PF} is the full name and version of the package, including the revision number — yes I know that it would have been better to use ${CATEGORY}/${PF} but let’s not go there for now). To do so you either use the Portage-provided dodoc and dohtml helpers, or you have to tell the configure script or the install makefiles to change the directory they install their documentation into; the defaults are never right because at the worst case they don’t take the version into consideration and at the best have no idea about the Gentoo-specific revision.
It doesn’t seem like a controversial rule to enforce, does it?
Well it seems like some developer (whom I’m not going to out just yet) doesn’t like that idea, and even if it’s now very well known that the bugs I’m opening from the tinderbox are vastly on behalf of QA decided to close two of my bugs as WONTFIX. And yes that pisses the hell out of me. Even more because the packages installing in the wrong directory do so because their upstream-provided source packages has been voluntarily split to provide a temporary counter-measure to the lack of proper USE dependencies (and this has caused more than a little grief I’d say).
But it’s not the first time I get responses on this line; for instance some developers find too bothersome to explicitly list the upstream-provided precompiled (sometimes proprietary sometimes not) files that fail important QA/security checks like RPATHs and TEXTRELs, and thus prefer simply restricting binary checks entirely (note: binchecks also fixes vulnerable and wrong RPATHs such as empty entries or current-work-directory).
We got all the “let’s take shortcuts” developers who prefer to commit half-thought stuff without masking it or without experimenting with it for a while to flesh out eventual requirements. While importing a huge amount of code that was developed outside of tree is always a risk (as the Ruby-NG effort is showing pretty well), simply pushing eclasses that support new versions of a language without allowing the user to disable them or without checking for proper reverse dependencies is a pretty bad idea.
Further on the problem comes with developers that drop patches without checking whether they are needed, and without warning that the problem might re-surface (for instance I know that I half-blindly dropped some patches from Linux-PAM; on the other hand I’m currently working on making it work on uClibc again).
So in general, yes, the problem with Gentoo’s QA is mostly developers, and yes, I know wee should come up with some rulebook that is “there or nothing”, I’ll see what I can do about that with the time I got.
Exactly right. That’s why I’ve just moved away from Gentoo, I got so sick of an update breaking core packages. Too damn tired to be fixing my system up every time I did an emerge world.
Right on! One of the best things about Gentoo is its QA – as a goal at least, even if the reality doesn’t always measure up. I know that if I ask an Ubuntu dev a build-related question, I’ll be told a way that will “probably work”, but if I ask Gentoo, I’ll get a thorough answer guided by knowledge of a lot of other packages. The answer might be a bit more complex than I’m ready to implement at the moment, but at least I know what the “right” answer is!I don’t think people like Phil realize just how difficult it is to ensure quality on a source-based distro like this. You’ve blogged quite a lot about the difficulty of testing just a few combinations of packages, versions, USE flags, etc., and even if the Tinderbox were unimaginably fast, it would still not be able to test a fraction of the configurations that users will see.That said, there’s definitely room for improvement in the Gentoo development processes, and you’ve outlined one such area here.
No need to hide my identity. I closed those bugs as won’t fix, because I thought it was trivial. I would expect most people to look for /usr/share/doc/qt-$version, if not immediately, then after failing to find a directory for the specific module.I am not aware if this is a documented policy, but if it is, please point me towards the location, as I can’t find it. All I have been pointed to so far is documentation on dodoc’s behaviour; not anything that requires that location be used without exception. Also, the qt docs location has been used since we started using split ebuilds, and I have never before heard a complaint.And I would find it a common courtesy to talk to me directly before venting on your blog. If we approach each other in a friendly and rational manner, we should be able to resolve our differences.
The Ebuild HOWTO in the developer handbook says (under ‘File System Locations’) “Documentation files are installed in /usr/share/doc, into a subdirectory reflecting the name, version, and revision of the particular program.” I assume this is what Diego is referring to.While this doesn’t actually mandate /usr/share/doc/${PF} it seems to eliminate anything else that would be a logical choice.
you’re dead on the mark
devs are not paid but working willingly.thus while most try to do their job properly (and some like to have it proper) you can’t require them to do alllll the time extra work, it doesn’t work.sure is a problem but to solve it will be difficult.i hope this entry will still make some aware it would be nice to do a bit more checking
William: Thank you for doing what the QA team was unable or unwilling to do. I will now proceed with working out the best solution.
If I compare the things I had to go through to try and get an account on sunrise and then see that official tree developers don’t even check the correctness of their Manifest or the correct name of a patch…
@Dustin – yes I do realize. I used Gentoo for 4 years. I’m a very technical computer user (although I don’t code). I’m a Linux and Windows server admin in my job.My point is, I got tired if every X update breaking X. I got tired of package abc needing package xyz, which then would fail to compile so I had no easy way to get abc installed. This has happened more and more over the past 12 months. Things like KDE 4 staying hard-masked way beyond what was necessary, and yet other apps going into the stable tree which wouldn’t even compile. THAT is my issue with Gentoo. I know what goes into building the distro, I know it’s free. But Fedora is also free (as is Slack, Vector Linux, etc etc) and they also have community-provided packages that don’t seem to constantly fail.Tell me, why should I put up with having a fully functioning system, then an emerge world pulls down an updated X – then when that’s installed, I have no mouse and keyboard anymore? It worked FINE before the update. Just how are you supposed to fix X when you’re logged into your desktop and you’ve no mouse or keyboard? You have to hard boot your system (possibly damaging the file system), boot up to init 3, then try to access the forums through lynx to see if it’s a known issue, blah blah. I don’t have time for that.
Phil–Hope you have good experiences with whatever system you’ve switched to. Personally, I can’t remember the last time X broke or a package in portage failed to build… I’ve been using Gentoo a bit longer than you, but I don’t know how the both of us could be using it so differently such that you had so many problems and I didn’t…I do test a wide variety of packages, yet the most annoying thing I’ve recently had to deal with was a block which had to be fixed manually… It was fixed with a simple emerge -C, and even those are getting quite rare…Regardless, I will readily acknowledge that several other distributions are also decent enough and you may very well feel better using one of them if Gentoo has not met your expectations. Good luck.
Diego,Great post! I find this one unusually well thought out & organized. That makes it much easier to read than some of the others, which can occasionally ramble on.
Aye, that is putting the finger in the sorest spot in all Gentoo. Kudos on that and hopefully this helps in decreasing annoyance from more casual users.@onefriedrice: really? you completely missed the change of a certain default about evdev in X? how it caused all non-evdev devices to be dropped, so keyboard and mouse no longer did anything?the one with AllowEmptyInput=no as workaround?I *still* haven’t been able to get alternate keyboard-layouts working again through config files, despite what the documentation says. the only thing that seems to work a bit are manual and the DE-specific stuff like in gnome and kde.xorg.conf is completely ignored and the xml-like configuration for the keyboard layout simple fails silently. only things failing silently are more annoying than package x/2 of x packages failing on a large update, causing other critical packages to fail merging as well.An emerge info notice giving a link on how to configure this in the new system would have done a LOT to alleviate annoyance, let alone wasted time in tracking down methods that work. it would *still* be helpful.A configuration that works uniformly in all DEs AND on the terminal (without X), of course.
Real QA folks(if they exists in any form more than just a rubberstamp on developers work) are ALWAYS the enemy of the developers, who hate to be told such and such nitpick is in the wrong place or such and such nigh-unto-impossible combination causes a crash, or “hey you failed to document your code”. Nothing unique here. :)That said, a QA nazi is important, if not popular, especially because some devs, even some very productive ones, are not fastidious about dotting i’s and crossing t’s. Its the least interesting of work
“Thank you for doing what the QA team was unable or unwilling to do. I will now proceed with working out the best solution.”I couldn’t say it better! Hope there will be more motivation int the team in future…
I have to agree with Phil. I’ve used Gentoo professionally for ca. 6 years now, and I’ve been seeing the portage quality drop very consistently. I installed Gentoo on almost any platform you can get your hands on in the past: Sparc, PowerPC, x86, embedded x86 including one real time acquisition device prototype on FPGA ISA card, AMD64, Itanium, even some old Alphas. It was thrilling to see early releases of Gentoo to be more stable and reliable than Debian (this used to be true e.g. of the early Alpha releases). I know some of those installations still run in production as database platforms or in computational clusters even though I no logner administer them, but they have not been updated for long for fear of breakage. Nowadays every other Gentoo update will break any fairly complex system almost as a rule. It would really be difficult to convince me that the quality of Gentoo has not declined. Failing system updates used to be non-existent in Gentoo a few years ago. Failing world updates were real exceptions even with highly customized USE flags. The portage tree in 1.4 was almost as dependent as in FreeBSD ports collection. Currently I use a Gentoo desktop as a development platform at work and I run two Gentoo developoment boxes at home, and I can’t remember any larger update which has not failed due to blocking or circular dependencies or downright broken packages — at least a few times on the way. This is extremely counterproductive. As a user I’m not a system tester or a system maintainer: I need my platform to do my job. Because I need a relatively low-level and flexible Linux, I’m considering getting back to Slack now. I feel Gentoo developers prefer to have fun stacking still new complex features onto the portage functionality rather than maintaining a stable system, what a boring drudgery. Ironically, this is posted from a windows laptop — because both my home Gentoo systems are unusable after latest updates.
testing and enforcing QA is fine.Users reading emerge log outputs are fine too. Should there be an “emerge -v1 -j1 $(qlist -C x11-drivers/)” alike in xorg-drivers pkg_postinst() upon minor version change for those who fail in reading?Anyway, the text says nothing about the benefits of /usr/share/doc/${PF} over /usr/share/doc/${P} as docdir location? Since cd /usr/share/doc/”$(qfile -e -C -q /usr/bin/python2.6)” fails due to missing ${CATEGORY}, it’s nitpicking to insist on the revision at this place imho. Different revisions should not be subject to slotted installs, don’t know how debian handles this in their /usr/share/doc/${PN} scheme. But that should not be the point of discussion.Diego: Is there any particular good reason for /usr/share/doc/${PF} over /usr/share/doc/${P} besides, “it stands in Ebuild Howto” and “dodoc/dohtml do so” (which both can be changed)?