[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: How do parallel builds scale?
From: |
Ludovic Courtès |
Subject: |
Re: How do parallel builds scale? |
Date: |
Tue, 08 Mar 2011 11:23:53 +0100 |
User-agent: |
Gnus/5.110013 (No Gnus v0.13) Emacs/23.2 (gnu/linux) |
Hello,
Ralf Wildenhues <address@hidden> writes:
> * Ludovic Courtès wrote on Mon, Mar 07, 2011 at 06:16:17PM CET:
[...]
>> FWIW I think the cumulative plots make sense when trying to answer the
>> question “how many packages have a speedup <= X”.
>
> Well, if I'd ask a cumulative question I'd probably rather ask how many
> packages have a speedup >= X. Small but significant difference.
Hmm, OK.
>> >> > I am fairly surprised GCC build times scaled so little. IIRC I've seen
>> >> > way higher numbers. Is you I/O hardware adequate?
>> >>
>> >> I think so. :-)
>
> Can you please make your hardware specs available? I've had two 16-way
> systems, and one could barely get more than a speedup of 2, while the
> other cucked away nicely at 10 or so.
See
<https://plafrim.bordeaux.inria.fr/doku.php?id=plateforme:configurations:dancharia>.
[...]
>> >> > Did you use only -j or also -l for the per-package times (I would
>> >> > recommend to not use -l).
>> >>
>> >> I actually used ‘-jX -lX’. What makes you think -l shouldn’t be used?
>> >
>> > Typically, -lX leads to waves in the load, due to the latency between
>> > measure and action, and of course the retardation from the measure
>> > interval. There are long periods in which processes are already done
>> > but the load is still listed as too high.
>>
>> Right, I see.
>>
>> I don’t think it hindered scalability though, since as the measurements
>> show that few packages scale beyond 2, even with ‘-j32 -l32’.
>
> I don't believe this argument. I think -l is fundamentally flawed, and
> I don't see where the slowdown it causes should be bounded from below,
> except for the number of total compiles (which is higher than 32 in the
> case of GCC).
I’ll see if I can run the whole thing without ‘-l’. I’d be surprised if
all those builds suddenly scale up, though.
Thanks,
Ludo’.