Long title, hopefully catchy enough so that both users and fellow developers can get to read this, since it’s half a rant, half an explanation on why the QA team can be quite anal when it comes to bugs that, for most developers, and especially for the maintainers of the packages involved, might look minor or not causing problems to the general population of users.
Today’s problem for instance was with packages that, for non-live (thus, snapshot or release) versions used the SCM eclasses, fetching directly from CVS, Subversion, GIT, Mercurial and so on. QA already have stated that the ebuilds using SCM eclasses should be masked (it’s in the devmanual if you wish to look at it), and that by extension should tell that using them for relesed code is bad. Among the various reasons for not using SCM eclasses for proper versions of the software, no matter whether upstream has or hasn’t made any serious release, there are safety involvements (you sidestep the Manifest in portage) and problems with proxy (not all SCMs use HTTP for fetching), but what creates quite a bit of a problem to me with the tinderbox: those ebuilds don’t abide the fetch command! When I run the tinderbox, I launch in parallel both a build sequence and a fetch sequence (I cannot use the parallel fetch feature because I launch each package one by one); when the packages using SCM hit, they spend time not building, which is bad for the tinderbox. And it gets even worse when you add stuff like the old, removed rubinius that spent time timing out because the server went away.
But the same class of problems involve using too big files in the tree: when you add a 100K patch for a package that just a few users are going to merge, you’re wasting bandwidth and disk space for a huge amount of people who just don’t care. That’s why we stress the need for making filesdir as lightweight as possible without messing with the development process, of course. Again, this is just another little task for the developer: package the patches and send them on the mirrors, then add them to the ebuild so that they are only downloaded by those who do need them.
And again it’s the same problem with packages ignoring compilers, compiler flags, linker flags, or prestripping when they shouldn’t. These are a problem because one of the selling points of Gentoo is the customisability of it all, at all different levels. So while they might be seen as minor points, these are all the little details that the QA team has to answer for. And it goes on and beyond this, making sure software builds, that it builds in parallel if it’s possible, that it doesn’t fail with new version of dependencies, that it builds with the correct kernel.
So please help us helping everybody, and don’t just ignore our requests, or start a pissing contest on why your package should be special, and not abide to the common rules and directions of the rest of Gentoo. Sorry, but unless it is really special, and that’s pretty rare, your software will have one way or another to abide to those rules, and if I have to piss you off by forcing the decision as QA, then I will. I hope I won’t have to, though.
The problem is the conflict between what is fast and what is theoretically right. It seems to me that this shows problems with our current system making it overly difficult to automatically create and mirror tarballs of SCM-only releases. If there were an ebuild function for that with nearly instantaneous mirroring, I’d be all over it.
I just ran the following command for fun:find /usr/portage/ -size +50k -exec du “{}” +Biggest surprises were a 200k patch under openais, and a 300k Manifest(!) for texlive-latexextra.
the usage of SCM breaks a lot of things…I have a laptop, which I usually connect to the cabel for emerge –sync && emerge –fetch, then I place it back where I usually have it while building, which makes the SCM-ebuilds go fetch over my wireless…Also, they break portage@rebuild_live, which AFAIK was developed in portage-2.2 to rebuild -9999 and other moving targets, but stuff like myth (that at least before used svn for all its fetching needs) wanted to be rubilt at the same time.
Hi,Can you post this to gentoo-dev list so that developers are aware of the issue. This would help preventing future problems.
Also a problem when the only protocol allowed is http:// by a firewall that you cannot control.I support your decision in this action item. Thanks for your work.