Re: single-threading targets although make was invoked with -jX ?

[Eric Melski]

On 05/04/2011 10:20 AM, Eric Melski wrote:
> On 05/04/2011 09:47 AM, David Boyce wrote:
> > On Wed, May 4, 2011 at 11:20 AM, Per Jessen<address@hidden>   wrote:
> > > yeah, SYNCSH_SERIALIZE is equal to what I do today using this construct:
> > >
> > > (flock -s 200; some-command $^ $@) 200>/var/lock/some-lockfile
> > >
> > > It satisfies the single-thread requirement, but in massively parallel
> > > runs, several of these often end up waiting for each other.
> >
> > Could you rephrase this? I can't see a meaningful distinction between
> > "waiting for each other" and "submit one at a time". Either way
> > they're serialized, no?
>
> They are serialized either way, but the processes could be consuming
> other resources (memory, disk, database connections, etc).  In the
> "submit one at a time" model, you avoid tying up those resources needlessly.

Oops, should have had my coffee before responding. :)  I see now the 
synchronization occurs outside the invocation of the process, so the 
process won't be able to acquire any other resources, because it won't 
actually have started.

I suppose if you're looking at a really, really massively parallel 
system you might have concerns with swamping the process table with 
effectively idle processes though.

br,

Eric Melski
Architect
Electric Cloud, Inc.
http://blog.melski.net/