parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Queue system limitation


From: Thomas Bereknyei
Subject: Queue system limitation
Date: Sun, 19 Sep 2021 12:38:31 -0400

The limitation mentioned in the man page has bitten me several times
until I remember. A quick `-j1` can get me around it for small things,
but that isn't what I want to express. I've attempted to use various
combinations of block/n/N/L options with no satisfactory solution. Is
there a way to remove this limitation altogether? I can warm-up the
jobslots with `(seq 1 16 ; job_generator ; ) | parallel --lb echo {}`,
and make those null jobs in the beginning have no-effect, but this has
come up enough I was hoping to find a better solution.

This is not referring to the limitation of the output visibility,
controlled by ungroup/line-buffer.

docs patch (hopefully isn't needed because we can remove the whole paragraph):

diff --git a/src/parallel.pod b/src/parallel.pod
index 7a8a9c22..18186bd1 100644
--- a/src/parallel.pod
+++ b/src/parallel.pod
@@ -4799,7 +4799,7 @@ In some cases you can run on more CPUs and
computers during the night:
 GNU B<parallel> discovers if B<jobfile> or B<~/.parallel/sshloginfile>
 changes.

-There is a a small issue when using GNU B<parallel> as queue
+There is a small issue when using GNU B<parallel> as queue
 system/batch manager: You have to submit JobSlot number of jobs before
 they will start, and after that you can submit one at a time, and job
 will start immediately if free slots are available.



reply via email to

[Prev in Thread] Current Thread [Next in Thread]