EAPI=4, missing tests, and emake diagnostics

Some time ago, well, quite a bit of time ago, I added the following snippet to the tinderbox’s bashrc file:

make() {
    if [[ "${FUNCNAME[1]}" == "einstall" ]] ; then
        emake -j1 "$@"
    else
        eqawarn "Tinderbox QA Notice: 'make' called by ${FUNCNAME[1]}"
        emake "$@"
    fi
}

I’m afraid I forgot whose suggestion that was, but it was an user, Edit: thanks to the anonymous contributor who actually remembered the original author of the code. And sorry Kevin, my memory starts to feel like it’s an old flash.

Kevin Pyle suggested me this code to find packages that ignored parallel build simply by using the straight make invocation over the correct emake one. It has served me well, and it has shown me a relatively long list of packages that should have been fixed, only a handful of which actually failed with parallel make. Most of the issues were due to to simple omission, rather than wilful disregard for parallel compilation.

One of the problems that this has shown is that, by default, all the make calls within src_test() default functions are not running in parallel. I guess the problem is that most people assume that tests are too fragile to run in parallel, but at least with (proper) autotools this is a mistake: the check_PROGRAMS and other check-related targets are rebuilt before running tests, and that can be done with parallel make just fine. Only starting with automake 1.11 you have the option to execute tests in simultaneously, otherwise the generated Makefile would still call them one by one, no matter how many make jobs you requested. This is why most of my ebuilds rewrite the src_test() function simply to call emake check.

This is the set-up situation. Now to get to the topic of the post… I’ve been seeing in the tinderbox some strange output, when looking at the compilation output, that I couldn’t find in the logs either. Said output simply told me that emake failed, but there was no obvious failure output from it, and the package merged fine. What was the problem? Well, it is a bit complex.

The default src_test() function contains a bit of logic to identify what the Makefile’s test target is, so that it can be used both with autotools (the default target to run tests with an automake-generated Makefile is check) and with otherwise-provided Makefile that likely use test as their testing target. This has unfortunate implications already, especially for packages whose maintainers didn’t test with FEATURES=test enabled:

• if the package has a check target that doesn’t run tests, but rather checks whether the installed copy is in working state, then you have a broken src_test(); qmail-related packages are used to this, and almost all the ebuilds for said packages never restricted or blocked tests;
• if the package does not have any specific test or check target, but it’s using make’s default rules, it could be trying to build a test command out of a non-existing test.c source file
• in any case, it requires make to parse and load the Makefile which could be a bit taxing, depending on the package’s build system.

To this you can now add one further issue, when you join the above-quoted snippet from the tinderbox’s bashrc and the new EAPI=4 semantics for which ebuild helpers die by default on failure: if there is no Makefile at all in $S then you get an emake failure, which of course wouldn’t be saved in the logs but would still be repeated at the end of merge just for the sake of completeness. Okay, it’s just a nuisance and not a real issue, but it still baffled me.

So what is my suggestion here? Well, it’s something I’ve been applying myself independently of all this: declare your src_test explicitly. This means not only add it with the expected interface even if tests are present, but actually null it out if there are no tests: this way no overhead is added when building with FEATURES=test (which is what I do in the tinderbox). To do so, rather than using RESTRICT=test (which at least I read as a “tests are supposed to work but they don’t”), do something like:

src_test() { :; } # no tests present

I usually do the same for each function in virtual packages.

I really wish that more of my fellows started doing the same, the tinderbox would probably have a much lower overhead when not running tests, which seems to be an unfortunate scenario, as most automake-based packages will try a pointless, empty make check each time they are merged, even if no test at all is present.

I’ll follow up with more test-related issues for the tinderbox in the next weeks, if time permits. Remember that the tinderbox is on flattr if you wish to show your appreciation… and I definitely could use some help with the hardware maintenance (I just had to replace the system’s mirrored disks, as one WD RE3 died on me).