Today lurking on #gentoo-hardened I came to look at an ebuild written particularly badly, that exasperated one very bad construct for what concerns parallel builds (which are a very good thing with modern multi-core multi-thread CPUs):
src_compile() {
if use foo; then
emake foo || die
fi
if use bar; then
emake bar || die
fi
if use doc; then
emake doc || die
fi
}
This situation wastes a lot of processing power: the three targets with all their dependencies will be taken into consideration serially, not in parallel; if you requested 12 jobs, but each of foo and bar only have three object files as dependencies, they should have been built at the same time, not in two different invocations.
I admit I made this mistake before, and even so recently, mostly related to documentation building, so how does one solve this problem? Well there are many options, my favourite being something along these lines:
src_compile() {
emake
$(use foo && echo foo)
$(use bar && echo bar)
$(use doc && echo doc)
|| die "emake failed"
}
Of course this has one problem in the fact that I don’t have a general target so it should rather be something more like this:
src_compile() {
local maketargets=""
if use bar ; then
maketargets="${maketargets} bar"
else
maketargets="${maketargets} foo"
fi
emake ${maketargets}
$(use doc && echo doc)
|| die "emake failed"
}
This will make sure that all the targets will be considered at once, and will leave make to take care of dependency resolution.
I tried this approach out in the latest revision of the Drizzle ebuild that I proxy-maintain for Pavel; the result is quite impressive because doxygen, instead of taking its dear time after the build completed, runs for about half of the build process (using up only one slot of the twelve jobs I allocate for builds on Yamato).
Obviously, this won’t make any difference if the package is broken with respect to parallel build (using emake -j1) and won’t make a difference when you’re not building in parallel, but why not doing it right, while we’re at it?
In the common case where the use flag matches the name of the make target, there is a shortcut available to simplify the command even more:
src_compile() { emake all $(use doc && echo doc) || die}can be replaced with
src_compile() { emake all $(usev doc) || die}The
usev()command has been in portage since about the dawn of time, so you don’t even need to worry about the EAPI.Diego,Wonder what will happen if neither of the 3 use flags are enabled. Will the ‘make’ degenerate into ‘make all’? If not, which mechanism prevents that?Also perhaps I am missing something obvious, but what is the difference with your last example, except that foo and bar order is changed?Ok, maybe the ‘general taget’ is a term that would explain my confsion, but both google and ‘info man’ fail to shed light on this…
Ah thanks Jonathan I entirely forgot about @usev@.Drybone, the last example ensures that foo is enabled if bar is not. In general, you should always have at least a target enabled when you start @emake@, so you might have to simply _default_ to one if the USE flags would tell you to have no target at all.
Ah, so in that example, unlike the previous ones, you can not have foo and bar both enabled simultaneously. Not the most usual thing for USE flags, which is a point of my confusion. And I afraid checking if default target should be added is not going to be the most elegant piece of code… well, maybe clumping the targets together into a string first, and then checking if it is still empty could work – still sounds hackish.So things would go much easier if each software package contains a target like ‘all’ or ‘bin’ which could be given to emake unconditionally. If that is not possible, then perhaps the package is a good candidate to be split into several more specific independent packages. If I’ve got the world model right at this time. 🙂
src_compile() { local maketargets=$(usev doc) if use bar ; then # use foo && ewarn "bar && foo both enabled, bar kill foo" # managed in src_prepare? emake bar ${maketargets} || die "emake failed" else emake foo ${maketargets} || die "emake failed" fi}