[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: --round-robin and --keep-order with --pipe
From: |
Ole Tange |
Subject: |
Re: --round-robin and --keep-order with --pipe |
Date: |
Mon, 12 Aug 2013 08:38:13 +0200 |
On Sat, Aug 10, 2013 at 11:50 PM, Ole Tange <tange@gnu.org> wrote:
> I am considering letting --round-robin become default for --pipe, and
> let --keep-order mean --not-round-robin.
>
> That will mean that the default will not start one job per block, but
> instead on job per CPU. Only when running --keep-order a job per block
> will be started - and the order of the output will be fixed.
>
> Can you find situations where this change of default will cause problems?
One case came up: If GNU Parallel is used as a filter for an infinite stream:
cat /dev/urandom | parallel --pipe cat | running_statistics
If --pipe defaulted to --round-robin, GNU Parallel would have to cache
the full output of all the 'cat's causing $TMPDIR to run full.
So that seems like a bad idea.
/Ole