[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: GNU Parallel seems to drop
From: |
Dirk Eddelbuettel |
Subject: |
Re: GNU Parallel seems to drop |
Date: |
Tue, 25 Sep 2012 04:42:48 +0000 (UTC) |
User-agent: |
Loom/3.14 (http://gmane.org/) |
Here is more concrete example:
$ wc -l data.txt
1000000 data.txt
$ cat data.txt | awk -v path=dataSerial '{print $0 > (path "/" $1 ".txt")}'
$ cat dataSerial/*txt | wc -l
1000000
$ cat data.txt | parallel --pipe -- awk -v path=dataParallel -f script.awk
edd@max:/tmp/parallel$ cat dataParallel/*txt | wc -l
50040
edd@max:/tmp/parallel$
edd@max:/tmp/parallel$ cat script.awk
{
print $0 > (path "/" $1 ".txt")
}
edd@max:/tmp/parallel$
Now, am I being silly and just falling into the trap of a race condition
where my data is being lost when written?
Dirk
- GNU Parallel seems to drop, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop,
Dirk Eddelbuettel <=
- Re: GNU Parallel seems to drop, Ole Tange, 2012/09/25
- Re: GNU Parallel seems to drop, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop, Ole Tange, 2012/09/25
- Re: GNU Parallel seems to drop, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop, Ole Tange, 2012/09/25
- Re: GNU Parallel seems to drop data, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop data, Ole Tange, 2012/09/25
- Re: GNU Parallel seems to drop data, Dirk Eddelbuettel, 2012/09/25
- Re: GNU Parallel seems to drop data, Ole Tange, 2012/09/25
- Re: GNU Parallel seems to drop data, Dirk Eddelbuettel, 2012/09/25