RE: Parallel Question Issue

[Hogue, Robert (Nokia - CA/Ottawa)]

Hi,

Thank you for the response, I set the logfile for the script. 

exec >/var/www/html/rbim/scripts/results/"$output_file_name"_2 2>&1

And parallel runs inside the script.

Should I not do that and specify the output just for parallel like you 
mentioned.

I set the timeout because I ssh to a system and if the system is not up or gets 
stuck waiting for login I need to have it timeout and move on.

As I mentioned I can run it for 500 ips and all is good but when I run it on 
2000 ips the ones that passed with 500 not show some timing out and not 
providing the command output I send with ssh.

Thanks for your help.

Should I use other different options?

Bob


-----Original Message-----
From: Ole Tange <ole@tange.dk> 
Sent: Friday, March 5, 2021 4:13 PM
To: Hogue, Robert (Nokia - CA/Ottawa) <robert.hogue@nokia.com>
Cc: parallel@gnu.org
Subject: Re: Parallel Question Issue

On Thu, Mar 4, 2021 at 11:22 PM Hogue, Robert (Nokia - CA/Ottawa) 
<robert.hogue@nokia.com> wrote:

> I am running the below parallel when I run it again 500 targets in this case 
> commands that ssh to host and grabs information.
>
> I runs fine.
>
> parallel --timeout 800% --tag linux_boxes_ssh :::: "$passwords" 
> "$file_with_targets"
>
> If I run it against 10,000 targets a lot of them do not right the results to 
> the log file.

Where is the log file mentioned? If it is part of linux_boxes_ssh then that 
explains the missing lines: You are writing to the same file in parallel.

Instead you should write to stdout (standard output) and do:

    parallel --timeout 800% --tag linux_boxes_ssh :::: "$passwords"
"$file_with_targets" > my.log

> What can I set to make sure it still runs and I need a timeout just in case 
> one of the jobs gets stuck I need it to time out.

--timeout 800% is a good idea. If you add --retries 10, then it will try 10 
times if it fails.


/Ole