[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Poor job distribution?
From: |
Fratkin, Allyn |
Subject: |
Poor job distribution? |
Date: |
Mon, 6 May 2013 18:16:43 +0000 |
First let me say thanks to Ole Tange for writing such a useful program.
I do have one issue that makes me wonder if I am doing something wrong or if it
is a bug in parallel.
If the number of jobs I run is less than the number of job slots (hosts * jobs
per host) then parallel seems to do poorly at distributing the jobs. I would
hope for an even distribution but I don't get that at all.
An example:
7 hosts, 3 jobs per host = 21 job slots
11 jobs
I would have hoped that each host got at least 1 job, with 4 getting two. But
here's what happened:
host 1: 3 jobs
host 2: 0 jobs
host 3: 2 jobs
host 4: 0 jobs
host 5: 3 jobs
host 6: 3 jobs
host 7: 0 jobs
If it seems crazy to have 21 job slots and only 11 jobs, it's because I use a
fixed config of hosts and jobs slots and then feed it different sets (varying
numbers) of jobs.
Am I doing something wrong?
I'm using GNU parallel 20130422. The job distribution that parallel picks is
100% repeatable.
Reproduce with:
for x in `seq 1 11`; do echo sleep 30; done | ./parallel -j 3
-Shost1,host2,host3,host4,host5,host6,host7 --progress
Thank you.
--
Allyn Fratkin allyn@hp.com
Hewlett-Packard Company http://www.fratkin.com/
- Poor job distribution?,
Fratkin, Allyn <=
- Re: Poor job distribution?, Ole Tange, 2013/05/06
- RE: Poor job distribution?, Fratkin, Allyn, 2013/05/07
- Re: Poor job distribution?, Ole Tange, 2013/05/08
- tcgetattr: Invalid argument, Fratkin, Allyn, 2013/05/09
- Re: tcgetattr: Invalid argument, Martin Møller Skarbiniks Pedersen, 2013/05/09
- RE: tcgetattr: Invalid argument, Fratkin, Allyn, 2013/05/09
- Re: tcgetattr: Invalid argument, Ole Tange, 2013/05/10