[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: How to test the output of gnu parallel
From: |
Ole Tange |
Subject: |
Re: How to test the output of gnu parallel |
Date: |
Tue, 18 Sep 2012 22:50:36 +0200 |
On Tue, Sep 18, 2012 at 9:22 AM, yacob sen <yacob_123@yahoo.com> wrote:
>
> Dear All,
>
> I am trying to test if a file exists or not in a remote server. I have 4
> processors on my disposal in my laptop. Here is what I am trying to do:
>
> for value in {10..20}
> do
> #*************************************************************************************************
> seq -w 1 1 100 | parallel -j+0 wget ${ftp_site}/$value/{}/{}.F >& /dev/null
> && echo Y >> tmp || echo N >> tmp
>
> read test < tmp
> if [ $test = "Y" ];then
> echo "to an output file, "temp.$value.{}""
> fi
> done
> #*************************************************************************************************
Why only use 4 processes in parallel? If the webserver can handle it
you can spawn more:
parallel -j100 "wget -q ${ftp_site}/{1}/{2}/{2}.F && echo {1}/{2}"
:::: <(seq 10 20) <(seq -w 100) > outfile
> If the file exists in the remote server, the value in "tmp" should get "Y" ,
> in that case the file is saved to a file. My question is how to test the
> existence of all the 4 processors at the same time ?
The problem here is that you need a tmp file for each process -
otherwise it will be overwritten and thus you cannot predict by which
one, so avoid the tmp-files if you can.
/Ole