In my previous post I’ve written about my thoughts about developing a system to automatically produce report bugs out of a compilation failure from the tinderbox. While the thought was born out of a need in the tinderbox, after writing about it, I start to think it might actually be useful to implement in a much more generic scale, so that it can be used by users as well.
The idea is that if you can actually extract all the most important information about a merge failure in a single report, it’d be also possible to use that report to open a bug report on the bugzilla. Since you know the package name and version, you can also found the proper maintainer, making sure you’re using the latest version of the metadata file. Since you know the revision, you can also compare it with the currently-available version on some websites, so to stop the user from reporting a bug that might have been fixed already.
Furthermore, by filing bugs through an automatic or semi-automatic system you can also provide a standard summary line for bugs, or make use of the status whiteboard, so that the auto-filing feature can also check for eventual already-reported bugs (this would also turn out quite useful in the tinderbox).
The whole point of me writing down these ideas is that they might sound farfetched, and they did to me for a long while, but the truth is that you just need to break through the doubts, breach the barrier finding one place to start attacking it, and I think I did; starting from what I need for the tinderbox now, I really want to develop the idea of report files, and then start working with somebody, and with pybugz, so that the auto-filing feature can actually be tested somewhere. This is something that we could have by the end of the year, in my opinion.
But there is more to this; there are times when just knowing versions and USE flags don’t just work, for build failures you need to get some more build logs; for instance I already noted that econf produces a config.log, and that both epatch and the autotools.eclass functions create logs in the temporary directory of the package. While for these examples we could just wire the support up, there are many other packages that don’t use autotools, and thus won’t be helped by these. We could probably wire up CMake as well, since it is becoming widespread, but the rest?
Indeed there are packages with custom build systems, like FFmpeg, that have their own logs for the configure stage, and others, like a Python extension I sincerely forgot the name of, that hides the errors from the default build log and force you to pick up another file to find the actual errors (I find this pretty stupid, but that’s what you’re given to work with).
What I propose is then adding a new function that can either provide a list of files to attach, or even generate further content that is needed to be bale to file a bug for a build failure of that package (think about non-gcc compilers settings, like ghc and fpc, or the Java compilers). With that inside the ebuild you could really write up a complete report, and make bug filing an almost entirely automated process, which would mean more bugs filed by users (rather than the tinderboxes) and a much more pleasing tinderbox setup (given somebody can actually wrangle the bugs up for the latter case).
Anybody up to the task of helping me out here?
If you’re willing/able to wait til the beginning of next summer, I think this would be an excellent google summer of code project.
I think it would be a good idea to take a gander at what CPANts does. http://www.cpantesters.org/…It might be beneficial to have the option to report both successful and unsuccessful builds to a reporting service ( opt-in ) so you can quickly see how reliable something is quantitatively.The amount of value-add for tree-cleaners and whatnot if you could get it to show a temporal graph showing the average number of successes/week vs failures/week of a given ebuild, as well as the amount of quality assurance given for people who have to stabilise things. ( if you could show stability/arch/time )Though I’d recommend keeping it *out* of bugzilla, it’ll just limit you having them in it.I’d start with being able to report one of 3 states:1. Build Only, Success2. Build + Test, Success3. Build Failed4. Test Failed.Including a full copy of the install log. Its not comprehensive, but its a good first stage to attain.I’d probably prefer it if it were something that were supported in all PMS’s, so a proof-of-concept could be divined and then draw up a GLEP and pray somebody slates the feature for EAPI5 :)When you’ve got that much going, you can start doing custom-ebuild metadata extraction at various levels, because IMHO, just being able to report a “Builds/Doesnt builds” status and a basic idea of what you’re running is *immensely* useful.
Wouldn’t this lead to an overwhelming number of duplicate bugs?
I’m willing to help test. I have basic bash scripting skills and have been using Gentoo since 2003.
I run the FTBFS (Fails To Build From Source) process for Fedora. http://fedoraproject.org/wi…As such, I have written a small script which auto-submits bugs, including in them the build logs. It tags each as blocking a larger FTBFS bug (e.g. F12FTBFS), and checks that a bug for the same package does not already exist that likewise blocks. Yes, there are some duplicates, such as if a package fails to build across several OS releases, or if another bug exists but does not block the FTBFS bug, but those are relatively rare, and the package owners tend to close one or the other as a dupe.I’ve been doing this for a few years now, and it seems to work pretty well.
for maintainer lookup / bug assignment, you can use:http://overlays.gentoo.org/…