A far less common problem than the last two I have written about, today I wish to analyse the failure in media-gfx/sam2p I reported. I have found similar problems before, and thus I think it’s another case worth talking about although the fix is very quick.
The failure in question would be this one:
Created executable file: ps_tiny (size: 47530).
ps_tiny: error at 1.2.1: tag %
The “premature EOF” error message usually means a file is truncated. With experience, you can tell this is a race condition: either the same broken rule or two rules are creating and deleting a file, and one of the two is arriving after it was deleted already.
In this case, looking at the original Makefile, it’s not the same broken rule:
l1g8z.pst: l1zip.psm psmlib.psm ps_tiny
<$< >tmp.h perl -pe0
$(CXX) -E $(L1_FLAGS) -DUSE_A85D=1 tmp.h >tmp.i
tmp.pin $(PREPROC_STRIP)
tmp.ps0 ./ps_tiny
tmp.pst $(TTT_QUOTE) $@
mv -f tmp.pst $@
l1ghz.pst: l1zip.psm psmlib.psm ps_tiny
<$< >tmp.h perl -pe0
$(CXX) -E $(L1_FLAGS) -DUSE_HEXD=1 tmp.h >tmp.i
tmp.pin $(PREPROC_STRIP)
tmp.ps0 ./ps_tiny
tmp.pst $(TTT_QUOTE) $@
mv -f tmp.pst $@
l1gbz.pst: l1zip.psm psmlib.psm ps_tiny
<$< >tmp.h perl -pe0
$(CXX) -E $(L1_FLAGS) -DUSE_BINARY=1 tmp.h >tmp.i
tmp.pin $(PREPROC_STRIP)
tmp.ps0 ./ps_tiny
tmp.pst $(TTT_QUOTE) $@
mv -f tmp.pst $@
I didn’t copy over all the rules, but this already shows the problem here. All the rules, while not exactly identical (the flags passed to the pre-processors are different depending on the target), use the same setting and use the same file names. The result is that while one rule runs the others will run too, creating the race condition.
For Gentoo I fixed it in a slightly sub-optimal way, changing all the reference to tmp. to $@.tmp. This is not exactly the nicest way as the correct way would have been to create different rules that generate the various temporary stages, so that then they could be executed in parallel as much as possible, rather than only sequentially, but as I see very little space for parallelism here, and the build system is a bit of a mess, I thought it was much easier to leave it at that. The result is that the rules above would become:
l1g8z.pst: l1zip.psm psmlib.psm ps_tiny
<$< >$@.tmp.h perl -pe0
$(CXX) -E $(L1_FLAGS) -DUSE_A85D=1 $@.tmp.h >$@.tmp.i
<$@.tmp.i >$@.tmp.pin $(PREPROC_STRIP)
<$@.tmp.pin >$@.tmp.ps0 ./ps_tiny
<$@.tmp.ps0 >$@.tmp.pst $(TTT_QUOTE) $@
mv -f $@.tmp.pst $@
l1ghz.pst: l1zip.psm psmlib.psm ps_tiny
<$< >$@.tmp.h perl -pe0
$(CXX) -E $(L1_FLAGS) -DUSE_HEXD=1 $@.tmp.h >$@.tmp.i
<$@.tmp.i >$@.tmp.pin $(PREPROC_STRIP)
<$@.tmp.pin >$@.tmp.ps0 ./ps_tiny
<$@.tmp.ps0 >$@.tmp.pst $(TTT_QUOTE) $@
mv -f $@.tmp.pst $@
l1gbz.pst: l1zip.psm psmlib.psm ps_tiny
<$< >$@.tmp.h perl -pe0
$(CXX) -E $(L1_FLAGS) -DUSE_BINARY=1 $@.tmp.h >$@.tmp.i
<$@.tmp.i >$@.tmp.pin $(PREPROC_STRIP)
<$@.tmp.pin >$@.tmp.ps0 ./ps_tiny
<$@.tmp.ps0 >$@.tmp.pst $(TTT_QUOTE) $@
mv -f $@.tmp.pst $@
The alternative using pipes, for the first rule, would probably be something like:
l1g8z.pst: l1zip.psm psmlib.psm ps_tiny
perl -pe0 < $< |
$(CXX) -E $(L1_FLAGS) -DUSE_A85D=1 |
$(PREPROC_STRIP) |
./ps_tiny |
$(TTT_QUOTE) $@ > $@
I haven’t changed it into this because I didn’t have too much time to look into how much difference it makes, or to test it; I’ve written it down to my TODO list for the future, maybe it is a possible improvement.
In general, for parallel make, pipes should be preferred to temporary files, and if temporary files are needed, they should have a different names for each target, so that they won’t overwrite one the other when make is run in parallel.
