[Bug gas/32254] Build problems related to doc/asconfig.texi

[rdiez-2006 at rd10 dot de]

https://sourceware.org/bugzilla/show_bug.cgi?id=32254

--- Comment #4 from rdiez-2006 at rd10 dot de ---

> Are you running make with the "-j" option specified ?
> If so, does the problem go away if you omit this option or use "-j 1" ?

I tried without "-j 9", and the "make" step still generates doc/asconfig.texi
once. However, "make install-strip" afterwards generates that file only once,
instead of twice.

Therefore, there seems to be a race condition with the install-strip target
when using parallel builds. Generation of doc/asconfig.texi is not serialised
properly in the makefile.

In the past, when we only had conventional spinning disks, it was considered a
bad idea to run install targets in parallel. However, if you have SSDs, or
high-latency networks, a parallel installation may be faster.

Users shouldn't really need implement a workaround like the "find . -name
*.info -exec touch {}" you suggested.

I would fix that bug first. Unfortunately, I do not know enough about Binutils'
build system to do that myself.

The second bug I would fix is the needlessly regeneration of doc/asconfig.texi
in the install-strip target. If the first "make" step has generated the file
once, it shouldn't be regenerated again in the second install-strip step.

It is hard to understand Binutils' build system. There are very few comments,
and the autotools-generated files are also checked into the repository (like
Makefile.in), which I consider to be bad practice.

Nevertheless, I tried, and I tracked the generation rule down to
binutils-gdb/gas/doc/local.mk :

%D%/asconfig.texi: %D%/$(CONFIG).texi %D%/$(am__dirstamp)
        $(AM_V_at)rm -f %D%/asconfig.texi
        $(AM_V_GEN)cp $(srcdir)/%D%/$(CONFIG).texi %D%/asconfig.texi && touch
-m -r $(srcdir)/%D%/$(CONFIG).texi %D%/asconfig.texi
        $(AM_V_at)chmod u+w %D%/asconfig.texi

I think there are several issues with that rule.

The file deletion upfront is probably what makes the race condition visible.
But the whole generation strategy is wrong: if there is an error half-way while
copying the asconfig.texi, or in the chmod command, then the build system will
leave a corrupt file behind.

The right strategy is to copy the file to a different filename first, like
asconfig.texi.tmp, adjust the permissions etc. on it, and then move the file to
its final filename. The move is an atomic operation on the filesystem. That is
actually the only way to guarantee robust error handling in a makefile.

Instead of running chmod as a separate step, I would copy the file with: cat
%D%/$(CONFIG).texi > %D%/asconfig.texi

I am guessing that this rule just selects the full documentation, or a short
version of it, based on the value of $(CONFIG). I suspect that setting the
target file to the same date as the source file is wrong. This means that, if
$(CONFIG) changes, the date of the generated doc/asconfig.texi may move to the
past or to the future. I would say that generated files should have the
timestamp set to the generation time. I mean that the 'touch' command should be
removed.

The dependency on a directory timestamp "%D%/$(am__dirstamp)" is iffy. In a
makefile, files should generally depend on files, and it should not be
necessary to use timestamping tricks. But perhaps you know the reason?


> Well it may be something peculiar to documentation files, but my understanding
> was that for GNU make if file A depends on file B and they
> both have the same timestamp then the rule to build A will be executed,
> so that it can be guaranteed that A is newer than B.

I don't think that is how GNU make works. Its documentation states: "if any
prerequisite is newer than the target, then the target is considered
out-of-date and must be rebuilt". I couldn't find exact details, but I would
say that, if the timestamps match, the file is not rebuilt.

I believe that including generated files in source tarballs is bad practice.
However, the GNU coding standards actually kind of encourage you to ship
generated .info files:

-------8<-------8<-------8<-------
Naturally, all the source files must be in the distribution. It is okay
to include non-source files in the distribution, provided they are
up-to-date and machine-independent, so that building the distribution
normally will never modify them. We commonly include non-source files
-------8<-------8<-------8<-------

There has been at least once discussion about this in the past, because it
wasn't not correctly implemented, and that has caused some trouble:

https://sourceware.org/pipermail/binutils/2005-July/043031.html

In my opinion, the main problem is that, when uncompressing the source files
inside a tarball, the timestamps are effectively random. If you follow the GNU
coding standards (or maybe established practice), the build system has to
rebuild the .info files if the tools are available, but it cannot rely on the
timestamps. I guess that ugly workarounds were put in place in the autotools to
implement that behaviour.

If I were to ship pre-built .info files, I would place them into a separate
"prebuilt" directory. If the info tools are there, I would build the .info
files. If not, I would take them from the "prebuilt" directory. But I wouldn't
ship .info files next to their corresponding sources as if they had been
previously built by the build system on this machine. You wouldn't need to
worry then about timestamps.

I'm not claiming I know this subject 100 %. If you have a better understanding
of these issues, I would welcome your feedback.

-- 
You are receiving this mail because:
You are on the CC list for the bug.