Today, Markus brought to my attention bug #247219, which I reported myself, against dev-util/bugle. It seemed to me like it was a parallel build failure, skimming at the log when I reported the issues, but it turns out not to be related to that. On the other hand, there is a parallel make issue there, which only works out of sheer luck of not hitting race conditions, so it might be a good idea to look into it nonetheless. Since it’s simpler to document these cases for the public than having to explain the same situation multiple time, I promised him I would blog about it, so here it is a new episode of the For A Parallel World series .
When you look at the build log, you can see there are multiple calls to mkdir and multiple regeneration of sources between it, which calls out to an issue already described with make rules and the way they are written:
/bin/mkdir -p include/budgie
/bin/mkdir -p include/budgie
/bin/mkdir -p include/budgie
/bin/mkdir -p include/budgie
/bin/mkdir -p include/budgie
budgie/budgie -I. -T `test -f src/data/gl.tu || echo './'`src/data/gl.tu ./bc/gl-glx.bc
budgie/budgie -I. -T `test -f src/data/gl.tu || echo './'`src/data/gl.tu ./bc/gl-glx.bc
budgie/budgie -I. -T `test -f src/data/gl.tu || echo './'`src/data/gl.tu ./bc/gl-glx.bc
budgie/budgie -I. -T `test -f src/data/gl.tu || echo './'`src/data/gl.tu ./bc/gl-glx.bc
budgie/budgie -I. -T `test -f src/data/gl.tu || echo './'`src/data/gl.tu ./bc/gl-glx.bc
This is not making anything fail, but it’s certainly making the system do more work than it should since it’s executing the same commands five times rather than once. Since the script does seem to produce the sources one by one but rather as a monolithic request, we cannot apply the same solution we used for the past case (breaking down the rules so that they build a single file each), so we have to serialise the build a bit to make sure it is executed just once.
To do so, the easiest way is to interpose between the final sources and the original files another rule, that would be called just once and that would be able to take care of rebuilding the files just when needed. This is usually done by creating a timestamp file, so to make sure that the rebuild hits as soon as the original files are changed, and that it stops it from running more than once if the files have been generated already.
Let’s see the original rules then:
$(BUDGIE_BUILT_SRCS): budgie/budgie$(EXEEXT) src/data/gl.tu $(budgie_all_bc_files)
$(mkdir_p) include/budgie
budgie/budgie$(EXEEXT) -I$(srcdir) -T `test -f src/data/gl.tu || echo '$(srcdir)/'`src/data/gl.tu $(budgie_main_bc_file)
As usual, there are multiple targets to the rule, even if the command is just the same and does not vary depending on the file that is executed. So we change the files to depend on a new timestamp file, and use that to create the files:
$(BUDGIE_BUILT_SRCS): built-sources-ts
built-sources-ts: budgie/budgie$(EXEEXT) src/data/gl.tu $(budgie_all_bc_files)
$(mkdir_p) include/budgie
budgie/budgie$(EXEEXT) -I$(srcdir) -T `test -f src/data/gl.tu || echo '$(srcdir)/'`src/data/gl.tu $(budgie_main_bc_file)
touch $@
Now make will decide that for each of the files in BUDGIE_BUILT_SRCS it needs the built-sources-ts timestamp file, and thus call the actual generation rule once to create it, which incidentally creates the rest of the files.
The only remaining issue is to make sure that when you clean up the sources you also remove the file, so just scroll further down the Makefile.am and you can easily spot where to change it so that there is this line:
CLEANFILES =
$(BUDGIE_BUILT_SRCS) built-sources-ts
The trick here was to add the timestamp file together with the files declared in the variable so that they are cleared too. Now with a new build you’ll see just one mkdir and script call instead of five, less work for the build machine.
