[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: GNU Parallel seems to drop
From: |
Ole Tange |
Subject: |
Re: GNU Parallel seems to drop |
Date: |
Tue, 25 Sep 2012 13:43:22 +0200 |
On Tue, Sep 25, 2012 at 1:22 PM, Dirk Eddelbuettel <edd@debian.org> wrote:
>> One way to solve that is to instead have the first invocation open A1,
>> B1 and C1 while the second writes to A2, B2 and C2. You can use {#} or
>> $PARALLEL_SEQ for that by writing to A{#} or A$PARALLEL_SEQ.
>
> Hm. Then I have ~ N x cores files, and need to aggregate those. Won't win that
> over straight awk use, I fear. Shucks.
I hear mawk is faster.
You might be able to do some trickery with fifos:
mkfifo A1 A2 A3
and then a command similar to cat that will read (non-blocking) from
A1 and when there is not longer a full record then read (non-blocking)
from A2 and when there is not longer a full record then read
(non-blocking) from A3, and then start over with A1 again until A1-A3
are closed.
nonblock-cat A1 A2 A3 > A &
I assume order is not important.
/Ole
- GNU Parallel seems to drop, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop, Ole Tange, 2012/09/25
- Re: GNU Parallel seems to drop, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop,
Ole Tange <=
- Re: GNU Parallel seems to drop, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop, Ole Tange, 2012/09/25
- Re: GNU Parallel seems to drop data, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop data, Ole Tange, 2012/09/25
- Re: GNU Parallel seems to drop data, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop data, Ole Tange, 2012/09/25
- Re: GNU Parallel seems to drop data, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop data, Ole Tange, 2012/09/25
- Re: GNU Parallel seems to drop data, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop data, Ole Tange, 2012/09/25