[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Combining --pipe and --shebang options
From: |
Ole Tange |
Subject: |
Re: Combining --pipe and --shebang options |
Date: |
Mon, 19 Nov 2012 20:26:44 +0100 |
On Sun, Nov 18, 2012 at 3:13 PM, Ole Tange <ole@tange.dk> wrote:
> On Fri, Nov 16, 2012 at 2:13 PM, Michel Samia
> <michel.samia@firma.seznam.cz> wrote:
>
>> is it possible to combine --shebang and --pipe options?
>
> It ought to be. And your script shows a good reason why. But currently
> it seems you cannot.
>
> I have created this bugreport https://savannah.gnu.org/bugs/index.php?37766
It actually is possible to combine --shebang and --pipe - but not the
way you want.
--shebang considers the rest of the file as data as if it was received
on -a or through stdin.
#!/usr/local/bin/parallel --shebang --pipe -k -j24 cat
a
b
c
d
What you want is to pass the rest of the file to python and let
parallel chunk up stdin to the python script.
I really like your idea, but it is clearly not a bug that it does not
work currently.
Your idea would be useful for any script (Shell, Perl, Python) which
can either process only one file or process stuff from stdin. Using
GNU Parallel your script can suddenly process many files/blocks of
data and in parallel.
I do not see we can change the behaviour of --shebang, but we can
invent a new option.
So it could be something like:
#!/usr/bin/parallel --shebang-program --pipe -k -j24 /usr/bin/python
(Please come up with a better name than --shebang-program)
This should accept data on stdin which should be chunked and passed to
the python program. So the program will be called like:
cat foo bar | my_program
or:
my_program foo bar
Without the --pipe:
#!/usr/bin/parallel --shebang-program -k -j24 /usr/bin/python
should work like:
parallel -k -j24 /usr/bin/python my_program {}
So:
my_program foo bar
(echo foo; echo bar) | my_program
should do the same as:
parallel -k -j24 /usr/bin/python my_program {} ::: foo bar
We should allow for putting options on the command interpreter. E.g:
#!/usr/bin/parallel --shebang-program -k -j24 /usr/bin/perl -p
Also without the --pipe the {} should work as expected:
#!/usr/bin/parallel --shebang-program -k -j24 /usr/bin/perl -p {}
{.}.out > {.}.log
Are there things I am not covering? Are there use cases that this will
not cover?
/Ole
- Combining --pipe and --shebang options, Michel Samia, 2012/11/16
- Re: Combining --pipe and --shebang options, Ole Tange, 2012/11/18
- Re: Combining --pipe and --shebang options,
Ole Tange <=
- Re: Combining --pipe and --shebang options, Michel Samia, 2012/11/20
- Re: Combining --pipe and --shebang options, Ole Tange, 2012/11/23
- Re: Combining --pipe and --shebang options, Michel Samia, 2012/11/27
- Re: Combining --pipe and --shebang options, Ole Tange, 2012/11/27
- Re: Combining --pipe and --shebang options, Michel Samia, 2012/11/27