[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: parallel hanging when used with pipe
From: |
Ole Tange |
Subject: |
Re: parallel hanging when used with pipe |
Date: |
Sat, 30 Mar 2013 16:50:54 +0630 |
That very much looks like a bug caused by a race condition.
Unfortunately I need it more reproducible to be able to fix it.
So I would like to ask you to try to reproduce it on a virtual machine
from: http://sourceforge.net/projects/virtualboximage/files/ using
data that you are allowed to give me.
/Ole
On 3/30/13, juncus@gmail.com <juncus@gmail.com> wrote:
> I was using parallel to do blast searches on a large number of
> biological sequences.
>
> cat sequences.fasta | parallel --block 2k --recstart '>' --pipe -j 30
> "blastn -db nt -task blastn -evalue 0.00001 \
> -outfmt '6 qseqid sseqid pident length mismatch gapopen qstart qend
> sstart send evalue bitscore ppos' \
> -max_target_seqs 5 -query -" > results.txt
>
> It hung after a few hours. Only about 40% of the input file had been
> processed. No blastn processes were running, and the parallel process
> was in sleep (S) state.
>
> I killed the parallel process and got this message:
>
> parallel: SIGTERM received. No new jobs will be started.
> parallel: Waiting for these 0 jobs to finish. Send SIGTERM again to stop
> now.
>
>
> This message seems somehow pathological. Parallel is waiting for zero
> jobs to finish. This must surely be connected with why it was hanging
> (it didn't get the message that jobs were done, so it didn't spawn new
> jobs.)
>
> This is not happening all the time, so difficult for me to diagnose. Any
> clues?
>
> Thanks very much,
> Owen
>
>
> $ parallel --version
> GNU parallel 20130222
> Copyright (C) 2007,2008,2009,2010,2011,2012,2013 Ole Tange and Free
> Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> GNU parallel comes with no warranty.
>
> Web site: http://www.gnu.org/software/parallel
>
> When using GNU Parallel for a publication please cite:
>
> O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
> ;login: The USENIX Magazine, February 2011:42-47.
>
>