Re: pkg() in 4.0.0-rc1 breaks building of some packages

[Carnë Draug]

On 10 March 2015 at 14:19, Juan Pablo Carbajal <address@hidden> wrote:
> On Tue, Mar 10, 2015 at 2:37 PM, Mike Miller <address@hidden> wrote:
>> On Tue, Mar 10, 2015 at 08:30:34 +0100, Olaf Till wrote:
>>> Thought I mention it here before making a regular bug report, since it
>>> might be release-critical.
>>>
>>> Private function configure_make() of pkg() now calls 'make' with the
>>> '--jobs' option, enabling parallel builds. This should only be done in
>>> cases in which certain design rules are followed in package
>>> Makefiles. Currently it breaks at least building the parallel package:
>>>
>>> I've  the following lazy, stupid, and inefficient rule in this
>>> Makefile, which of course should be changed, but which is nevertheless
>>> legal:
>>>
>>> %.oct: %.cc
>>>       @MKOCTFILE@ -s -v $< sock-stream.cc
>>>
>>> This causes sock-stream.cc to be compiled to sock-stream.o for each
>>> built oct-file. make --jobs .. builds the oct-files in parallel, so it
>>> happens that one job is just about to (over-)write sock-stream.o when
>>> another job uses it for linking, causing the linking to fail.
>>>
>>> Before I supply a (trivial) patch, it should probably be decided
>>> whether the --jobs option should be just removed or made an option of
>>> pkg(), defaulting to non-parallel building.
>>
>> There should be a couple of other ways to address this without changing
>> pkg.
>>
>> As the end user, you can call `setenv OMP_NUM_THREADS 1` in the Octave
>> shell before doing the pkg install, that should translate to --jobs=1.
>> So in this sense it is already an option that the end user has control
>> over.
>>
> This is good to know! but as you see, it is not cristal clear for
> everybody. I think pkg() could be more useful if it provided a
> solution...maybe just calling the line you just provided.
>
> [...]
>
>> In the package Makefile, you should be able to use the .NOTPARALLEL
>> special target to have your build ignore the --jobs setting.
>>

This is clearly a bug on the Makefile (or should make guess that it can't
be parallelized?).  Either way, I don't think we need to change anything
in Octave core.

If the only reason to default to -j1 or even to have yet another option in
pkg, is to workaround broken Makefiles, then I don't think we really need it.

The default should be to use all cores unless the user wants something else.
In such case, that choice makes sense to be applied to all computations,
not only the call to make and that option already exists.

Carnë