Re: file permissions on joblog

[Christian Meesters]

Hi,

not quite. Under SLURM the jobstep starter (SLURM lingo) is "srun". You 
do not do ssh from job host to job host, but rather use "parallel" as a 
semaphore avoiding over subscription of job steps with "srun". I 
summarized this approach here:

https://mogonwiki.zdv.uni-mainz.de/dokuwiki/start:working_on_mogon:workflow_organization:node_local_scheduling#running_on_several_hosts 
(uh-oh - I need to clean up that site, many outdated sections there, but 
this one should still be ok)

One advantage: you can safely utilize the resources of both (or more) 
hosts - the master hosts and all secondaries. How much resources you 
require depends on your application and the work it does. Be sure to 
consider I/O (e.g. stage-in file to avoid random I/O with too many 
concurrent applications, etc.), if this is an issue for your application.

Cheers

Christian

On 7/28/22 08:24, Ole Tange wrote:
> On Tue, Jul 26, 2022 at 2:49 AM Rob Sargent <robjsargent@gmail.com> wrote:
> > I'm using GNU Parallel behind not one but two intermediaries.  I submit to the first, the 
> > first wraps the jobs in a "slurm" script and submits the jobs to the second 
> > (the slurm host).
> >
> > The jobs run on the slurm host under the account of the first intermediary, 
> > username and group not accessible to me.  I would like to be able to view the 
> > --joblog file but its permissions are 640.
> I take it that the problem is similar to:
>
>     ssh otheruser@localhost "parallel --joblog foo ..."
>
> Try:
>
>      mkdir /tmp/foo; touch /tmp/foo/log; chmod 666 /tmp/foo/log
>      ssh otheruser@localhost "parallel --joblog /tmp/foo/log ..."
>
> If you have access to a database:
>
>      parallel -Sotheruser@localhost --sqlandworker
> pg://user:pass@host/db/tbl 'id;echo' ::: 1
>      sql pg://user:pass@host/db/tbl "select * from tbl;"
>
>
>
> /Ole