“Put it there and wait for users to break” isn’t a valid QA method

So it seems like Jeremy feels we’re asking too much of maintainers by asking them to test their packages. As I said in the comments to his blog, picking up poppler as a case study is a very bad idea because, well, poppler has history.

Historically, new upstream poppler releases in the recent times have been handled by guys working on the KDE frontends, when they suited their need, rather than being coordinated with other developers. This isn’t much of a problem for them because binary distributions have no problem with shipping two version of the same library on different ABI versions, and even building packages against different versions is quite okay for them — it definitely isn’t for us.

In turn, within Gentoo, in the recent times, poppler has been maintained by the KDE team; mostly by the one developer who, not for the first time, wanted to bump to 0.16.0 a couple of days ago, without adding a mask first and ensuring that all the packages in tree that use poppler would work correctly with it. Jeremy states that doing such a test would make ~arch akin to stable; I and I’m sure other of my colleagues, rather see it as “not screwing users up”.

First of all, it’s not like bumping something that you expect not having trouble; bumping grep and finding out that half the packages fail whose tarballs were built with Gentoo and a very very old version of our elibtoolize patches is unexpected. Finding out that, once again, the new minor version of poppler has API breakage is not surprising, it’s definitely expected, which means that when committing the version bump you can have either of two thoughts in mind “I don’t care whether GNOME packages break” or “Ooh shiny!”. In either case, you should learn that such thoughts are not the right state of mind for committing something to the tree. I said that before of other two developers, I maintain my opinion now.

And just to be clear, I’m not expecting that we spend months in overlays before reaching the main tree, I’m just saying that if you know it’ll break something, you should mask it beforehand, and give a heads’ up to the developers maintaining the other packages to due their job and fix it.

Now, in the comments of that post, Jeremy insists that factoring in my tinderbox is not an option for him, because it is “counting on a single developer cpu/time”. Right, because there is no other way to test reverse dependencies, uh? The tinderbox is meant to help the general situation, but it’s definitely not the only way; even though I’d have been glad to run it for poppler to find and report the problems, the task of checking its reverse dependencies is far from impossible to deal with for a single developer. There are a grand total of … thirty-nine (39!) packages with reverse dependencies on poppler! So it’s not a “can’t be done” situation, it’s a “can’t be arsed”. This also brings Jeremy’s ratio of “7/14000” packages with a problem to a more worrisome 739. See the difference?

Simply put, do your own homework: learn about the package you’re maintaining, try hard not to break reverse dependencies; if it happens that you break a package unexpectedly, hey, it’s ~arch, we can deal with it. But do not commit stuff that you know will break a good deal of the packages depending on yours. Especially if you’re not out there to fix them yourself.