[Reproduce-devel] [task #15262] Source code downloads and Job manager

[Mohammad Akhlaghi]

Update of task #15262 (project reproduce):

                  Status:               Postponed => In Progress            
        Percent Complete:                      0% => 80%                    
             Assigned to:                    None => mkhellat               

    _______________________________________________________

Follow-up Comment #28:

Fortunately today I had some time to inspect this work and try merging it.
Having logs for the downloads and builds is GREAT! Thanks a lot for doing the
job Mohammad-reza.

Some minor corrections were done
<https://gitlab.com/makhlaghi/reproducible-paper/commit/60cfe5647da> on the
branch (as fully described in the commit message). Then I imported the
`master' branch into this branch to have all the new features since this
branch diverged. 

All this work is now pushed onto my own fork's `makelog' branch, but its not
yet merged with the master branch as described below:
https://gitlab.com/makhlaghi/reproducible-paper/tree/makelog

However, while doing some tests, I noticed a big problem: if the building of a
program fails, the processing continues!

Until now, we solved this problem like this: we used `&&' between the main
building commands until the final target is built. Therefore, as soon as one
command failed, it wouldn't go onto the rest and it would stop. But here,
because we need to pipe the outputs to `tee' the final status of the command
is the returned value of the last command in the pipe (the `tee', which always
succeeds).

For example if the `./configure' fails, it will go onto `make', `make
install', and finally build the program's main target, while it shouldn't. 

One solution to this problem is when we know the system has GNU Bash: with the
`-o pipefail' setting (for example by running `set -o pipefail', we can tell
Bash to return a failed result if any of the pipe'e commands fail. However,
for the first set of software in the template (until Bash is built), we can't
assume the existence of Bash. 

I searched a lot for a solution, but I don't have much time and had to stop
after a few hours. In particular the `script' command seemed useful, but it
messed with the terminal and parallel building, causing a crash.

So the updated branch is now available on my fork until we can find a solution
to this problem. 

Having download and build logs is a wonderful feature, and we have done most
of the job, so I have marked it as 80% complete. I hope we can find a solution
to this problem soon and merge it into the `master' branch.

    _______________________________________________________

Reply to this item at:

  <https://savannah.nongnu.org/task/?15262>

_______________________________________________
  Message sent via Savannah
  https://savannah.nongnu.org/