Re: job race!

[Thomas Sattler]

Hi there ...

reading the question, I thought about something like this:

parallel -j0 "sleep {}; echo {}; sleep \$SECONDS;
  kill \$PARALLEL_PID; kill \$PARALLEL_PID" ::: {3..32}

I'm using "sleep $SECONDS" to make a job double its runtime
and than signal parallel. It works (at least) with bash.

But only the fastest jobs output is printed. Guess: All
the other jobs are still running, even the "finished"
ones still run their sleep, so there's no output.

And it is even worse: The other jobs are not properly
killed, which is obvious if you replace 'sleep' by,
for example, 'stress'. Just keep an eye on 'top':

parallel -j0 "stress -c 1 -t {}; echo {}; sleep \$SECONDS;
  kill \$PARALLEL_PID; kill \$PARALLEL_PID" ::: {3..32}

After a while I found a solution, but parallel needs some
extra help. I need to do the timeout and the signaling
from outside parallel:

touch flagfile; { SECONDS=0; while [ -f flagfile ]; do sleep 1;
  done; pkill parallel; sleep $SECONDS; pkill stress; } & \
    parallel -j0 "stress -c 1 -t {} &> /dev/null && echo {};
      rm -f flagfile" ::: {3..32} 2> /dev/null

It's a long one-liner, but thanks to the "\" you can simply
copy'n'paste the command above (assuming your mail program
displays it as four lines).

The two 'pkill's first ask 'parallel' not to start other
workers and kill all running workers that did not finish
within twice the winners time.

All other ways I tried only showed the first result, or no
results at all, or did not stop the other workers.

To be honest, the expected result of the command above was
3-6 or maybe 3-7 but I always get 3-9. Exchanging "pkill
parallel" and "sleep $SECONDS" reproducibly brings 3-8,
for a reason I do not understand.

Thomas