Re: build system rules & algorithms

[grischka]

Mike Shal wrote:
> ... The stat()ing is just a rough estimate of that portion of
> make's execution time. Is there a better way to benchmark it to figure
> out the bottlenecks?

Rough estimates are always fine too, of course.

Here is one from a test with TUP's "make_v_tup" case, 10000 files,
for the "nothing changed" case, times in seconds:

make                   11.42
make -r                 3.83
read makefile           1.76 (913431 bytes = 0.5MB/s)
git status              0.27
tup upd                 0.01

You see that "git status" is very fast to teil the changes, of course
not as fast as TUP which knows the changes in advance.  But I'd assume
that this is roughly the time needed to "stat" the files (on linux).

Obviously GNU make spends most of the time with trying built-in
pattern rules (you can see it with "make -d ...").  Of course the
test case does not require that feature (as most other makefiles
out there don't either).

If we disable built-in rules then the big part is reading the makefile.
(To measure I did just stick a "$(error eof)" at the end of it).
0.5MB/s is a solid value with room for optimization. (TCC reads and
compiles 20-30MB/s).

For the rest of 1.8 seconds (3.84 - 1.76 - 0.27) we can assume that
a part of it is for "processing the DAG in the alpha way".

However we just don't know how big that part is.  As opposed to "read
makefile" and "git status" it does not call out for external resources,
so even if it's O(10000), it might be very fast actually, maybe under
0.1 secs.

Which would still leaves 1.7 seconds in the dark.  It could be some
quirk with GNU make's internal directory cache but that is speculation.

Well, have to say that with the monitor and auto-dependency features
TUP gives a really magic impression at first look.  Just the input
language seems rather limited atm.  What about abusing GNU make as
a preprocessor for makefiles->tupfiles?  In experience the tools that
succeed are the ones with working from/to conversions.

--- grischka