Re: About .NOTPARALLEL

[Lane Schwartz]

On Wed, Jun 8, 2011 at 2:03 PM, Lane Schwartz <address@hidden> wrote:

>  On Sun, Apr 17, 2011 at 8:06 PM, Paul Smith <address@hidden> wrote:
>
>> On Sun, 2011-04-17 at 16:44 -0500, Peng Yu wrote:
>> > However, I think the syntax of .NOTPARALLEL can be improved a little.
>> > For example, I'd like the a.txt and b.txt are not generated in
>> > parallel with anything other .txt files, but c.txt and d.txt be
>> > generated in parallel, when '-j' option is used. Currently the
>> > following Makefile is essentially the same as the above one. May I
>> > suggest to add this syntax to make?
>> > .PHONY: all clean
>> >
>> > all: a.txt b.txt c.txt d.txt
>> >
>> > .NOTPARALLEL: a.txt b.txt
>> >
>> > %.txt:
>> >       sleep 2; touch $@
>> >
>> > clean:
>> >       $(RM) *.txt
>>
>> There are two issues here.  First, deciding on the syntax is one thing;
>> implementing it inside of GNU make is decidedly another thing entirely.
>> Currently make has a concept of parallelism, or no parallelism.  There's
>> no concept of "per-target parallelism".  Getting make's internal
>> algorithms to understand that some targets should not be built in
>> parallel, skipped, and come back to later when there's nothing else
>> running is not a trivial change.
>>
> Paul,
>
> I understand that implementing this change would be non-trivial. That said,
> I'd like to voice my support for this feature request.
>
> Just this week I found myself wanting this functionality. Most of the jobs
> launched by my make file can be run in parallel, but a non-trivial subset of
> them can't be.
>
> Eli's suggestion of using --load-average with -j is a good solution if the
> reason for specifying some jobs as nonparallel is to prevent overloading the
> CPU. But there are other use cases where that solution isn't the answer. In
> my case, there are issues related to the specific programs being launched
> that cause error conditions when certain jobs are run in parallel. My
> current workaround is to run make twice, building some targets (the ones
> that support parallel execution) first with -j, then run make again without
> -j to finish building the rest of the targets.
>
> Paul, your suggestion of using order-only dependencies is a good one, and
> it may solve my problem is some cases. But it would be pretty awkward to use
> in my current situation.
>
> That's my 2 cents, anyway. I can certainly live without Peng's
> proposed feature enhancement, but it sure would be nice to have.
>


Just re-read Peng's emails. My use case is slightly different. His proposed
syntax marks specific targets so that they can never be run in parallel with
ANY other target.

What I'd like to do is specify some set of targets (a b c d), and ensure
that only among that set, only one target is running at a time. I don't care
if the other targets in the make file execute in parallel, and I don't care
if those other targets execute at the same time as one job from my serial
set (a b c d).

Hmmm... this has me thinking. I wonder if this behavior could be implemented
in user space via a function that adds order-only dependencies, in
conjunction with exec and call. It would be ugly, but might be doable.

Cheers,
Lane