I understand. My issue with --transfer is that it interprets all parameters as files, while only some of them are.
Example:
$ echo "12\nAB" > foo.txt
$ echo "12\nAB" > bar.txt
parallel --sshlogin 1/$REMOTE_HOST --transfer grep {1} {2} ::: foo.txt bar.txt ::: "[0-9]+" "[A-Z]+"
parallel: Warning: [0-9]+ is not readable and will not be transferred.
grep: [0-9]+: No such file or directory
parallel: Warning: [A-Z]+ is not readable and will not be transferred.
grep: [A-Z]+: No such file or directory
parallel: Warning: [0-9]+ is not readable and will not be transferred.
grep: [0-9]+: No such file or directory
parallel: Warning: [A-Z]+ is not readable and will not be transferred.
grep: [A-Z]+: No such file or directory
At a higher level, what I am trying to do is to pass a log file - specific to each job - so that the script I run can check its existence and avoid repeating the experiment if the log file is there.
It is an intricate mechanism, but I am not sure how to do it within parallel otherwise, because parallel executes the command remotely.
Following, is a slight modification of the man entry that could avoid misunderstandings.
file will be transferred to each sshlogin before a jobs is started. It will be removed if --cleanup is active. The file may be a script to run or some common base data needed for the jobs. Multiple --bf can be specified to transfer more basefiles. The file will be transferred the same way as --transfer.
file will be transferred to each sshlogin before a jobs is started. It will be removed if --cleanup is active. The file may be a script to run or some common base data needed for the jobs. It is not intended for files specific to single jobs (i.e., dependent on parallel parameters); for such purpose, --transfer should be used. Multiple --bf can be specified to transfer more basefiles. The file will be transferred the same way as --transfer.