Since this seems to be a common misconception I wanted to write (once again, I think I did this before, too) about what QA actually means for Gentoo and why I’m spending so much time, effort, and – directly and indirectly – money on the tinderbox idea (remember that the maintenance of the system is up to me, and that it costs to replace disks, get bigger disks, … I also start considering upgrading the CPUs to see if I can reduce the time needed for the builds…).
As people following me on identi.ca already know, last night me and Daniel Robbins had a divergence of opinions on who has the higher QA. I am absolutely unconvinced by his arguments, that in my informed opinion are just mirror-climbing. By his replies, it seems like heshould have an higher QA standard because stuff always build, if needed with hacks like unsetting (not filtering, not counteracting, unsetting) user LDFLAGS.
I strongly disagree; as it’s often mocked up with the phrase “it builds, ship it!”, software that builds doesn’t really mean it works properly, as it should be clear by the multiple test failures that I found. And indeed for this reason, there are a series of QA tests in Gentoo that actually make stuff fail from building; it’s way too easy to “fix” the packages that fail because of that by disabling the safety measures we have implemented; yet that’s what Robbins says is the “quick fix” that improves his QA over ours.
I do agree on the fact that uses shouldn’t be seeing the failure if possible, that’s why I do use emake -j1 in ebuilds but I also do that with open bugs ; the idea is that once we know there is a problem we might temporary ignore it. But that works for stuff like parallel make or --as-needed; compiler errors, especially those related to _FORTIFY_SOURCE and similar, need to be fixed not worked around (thus why I don’t like using casts either and I request users to submit proper patches if they want).
There is a tight correlation between strictness and QA; and while Gentoo really doesn’t seem to achieve the proper QA and user experience all over the place (but I want to thanks Thilo, Samuli and Ryan for their huge efforts lately, and not just lately), we are improving our QA processes. And still they are better than Robbins’s in my opinion.
Oh and by the way this does not mean I like the Debian way of handling QA with their eternal releases (yes I know they are now trying to solve that by using time-based releases, that doesn’t make it nice anyway). This because we had quite a few packages that ere broken by Debian patches, or that Debian accepted just fine while being totally broken in code and build system (hfsplusutils anyone?).
And at the same time, I don’t really pretend that all software has the same QA levels; sometimes you have to break stuff around to be able to get some improvement for the others too, so I do accept projects like funtoo that want to try stuff without having proper QA processes or loosened QA levels. My only problem is when people like Robbins try to sell for QA what really is a bunch of mixed hacks around the place.
I think both are rigth, the problem that both are facing are different focus:Gentoo (represented by you in this case) claims to have a better QA that helps developers find and fix bugs more easy.Funtoo (represented by drobbins) claims that a better QA since user experience is less interrupted by break builds.The focus and final goal are different, and there is where both are rigth.In my experience using distributions build by drobbins the user experience exels, since he left Gentoo it’s quality (as my user perspective) has been degrading slowly to the point that I’ve to remove it from my servers and install Ubuntu on my desktop (I regret from that, but it’s true that the time I was expending keeping a basic gentoo system was to much for me).A test failing means that it’s not working the way it’s pretended to work, but if it’s does what the user expects it can be a higher QA that
My first impression is that you are approaching it from a different angle, but are essentially saying the same. The “user” should not suffer, but in the background things really need to improve.Problem is that I don’t really see a strategy how the last bit is intended to be done for Gentoo. Bugzilla to me appears to be one of the major “issues” for the “user experience” as due to its shear size it seems impossible to work with. Many bugs seem to stay there forever…. A bug that Daniel linked to in his last blog-post is a nice example: http://bugs.gentoo.org/show…I don’t recall you blogging about Funtoo previously, any strong feeling on the move to Git as it seems to help the way how you work as well?
You both get the wrong impression that QA == “User Experience”… that’s not the case. I agree we have a _shitty_ user experience, but we are going to get a strong QA, if it continues this way.
Finding bugs caused by CFLAGS (or similar) often results in better final software.Yes, maybe it won’t build with some user flags, maybe it will build but segfault everywhere.What if those are real bugs, somewhat hidden when using empty flags? This may result in corrupted data or security issues.Yes, this may be annoying for the user, but at least it’s visible.From a developper and a gentoo user point of view, I’d better solve the bug than work around it with ebuild hacks.The next time I’ll try to build the package by hand without portage around, it’ll be better for me.I think that solving things in ebuild is better for the distribution doing it, but it feels like it’s worse (at least, not better) for the community.There is the other thing about long term vs short term too…
I agree with Diego that a strong QA process is a major issue for anyone using a stable system. People using such want software that not only builds, but also works as expected.Problem is, that these people are most likely not the majority of Gentoo users. I have no numbers, but my impression is that most people using Gentoo choose to do so for the gain in flexibility and the more bleeding edge character of many packages. This implies that they are people with deeper technical knowledge then the average OS user, and tend to go even more bleeding edge by using unstable. Using unstable then again includes the expectation of having some QA problems with packages in question, and therefor giving less attention to smaller problems in the sector.That might explain the view of the first reply, who prefers a compiling feature rich systems with maybe minor flaws over a mature system with less, but working, features in stable and more breakage in unstable during the QA process.On the long term a stricter QA during stabilization benefits everyone, including upstream and through that even other distros, but short term people see more breakage in the Gentoo unstable build process, and wrongly interpret it as bad QA, while in fact it is exactly the opposite.
This episode just remember me when the ex gentoo-devel @philantrop shared his “super-uber-aller” QA experience with one of my projects (yeah, @philantrop was involved with my bashrcng project in the beginning) and he end-up injecting hacks, craps and trash without any good reasons; what happens at my project in the end? it was able to execute a hidden “rm -rf /*” on some (not so rare) circumstances, silently destroying Gentoo boxes that run it.After asking @philantrop the reasons for those nasty hacks he simply reply me: “they was necessary to improve QA”; obiovously that hacks was totally useless and did not fix the real problems that @philantrop was pretending to solve.I’m with you about QA as “quality” is nothing to confuse with “user experience” and most important is something that must not deals with “hacks and dirty workaround” a-la debian; QA = real solutions to problems.At Gentoo we have the resources and people to reach high levels of QA and I believe that in the long term this philosophy will wins, making Gentoo a more attractive distro.Unfortunately the actual trend of the FLOSS world seems to prefer hacks, workarounds and the KDE principles: “do it quick, release it quick, break it quicker and harder”; that scare me a lot!
I strongly agree with Diego, making ebuilds “work”, no matter the cost, and implying the software is OK just gives a false sense of quality. Some of the packages I frequently use, had to be unmasked, by doing so I know I’m taking a risk. However, if a masked package is unmasked just because it builds, I get a false sense of comfort here.The problem of such an approach is that the idea behind it is mostly appreciated only by people with a great interest in the technical side of the problem. Normal users just want the stuff to work. However, even outside the IT world, there are numerous examples in which a false sense of comfort has lead to disasters, big and small, whether it is in the world of banking, energy or even construction.This discussion also reminded me of the recent dispute of Alan Cox and Linus Torvalds (http://linux.slashdot.org/s…. I have to note I don’t know the exact details, but as far I understood, Cox’ code was neccesary from a QA point of view, but it broke some (buggy) software, which in turn seemed unacceptable by Linus.I don’t like it when software breaks, but I rather have a real solution, than some q&d hack. @Diego: I really appreciate your hard work on improving the QA of Gentoo, even if it means that it (temporarily) leads to a somewhat lower user-experience.
@equilibrium:Regardless of whatever you may think of Philantrop, I do remember Philantrop as one of the better KDE devs Gentoo had in a long time.I’m not sure why your personal disputes with someone have to be posted on someone elses blogs. Same goes for my comment of course. 😉