[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Combining --pipe and --shebang options
From: |
Ole Tange |
Subject: |
Re: Combining --pipe and --shebang options |
Date: |
Fri, 23 Nov 2012 23:07:50 +0100 |
On Mon, Nov 19, 2012 at 8:26 PM, Ole Tange <tange@it-pol.dk> wrote:
> On Sun, Nov 18, 2012 at 3:13 PM, Ole Tange <ole@tange.dk> wrote:
>> On Fri, Nov 16, 2012 at 2:13 PM, Michel Samia
>> <michel.samia@firma.seznam.cz> wrote:
>>
>>> is it possible to combine --shebang and --pipe options?
:
> Your idea would be useful for any script (Shell, Perl, Python) which
> can either process only one file or process stuff from stdin. Using
> GNU Parallel your script can suddenly process many files/blocks of
> data and in parallel.
:
> So it could be something like:
>
> #!/usr/bin/parallel --shebang-program --pipe -k -j24 /usr/bin/python
>
> (Please come up with a better name than --shebang-program)
It is now named --shebang-wrap since it wraps the shebang line.
> This should accept data on stdin which should be chunked and passed to
> the python program. So the program will be called like:
>
> cat foo bar | my_program
>
> or:
>
> my_program foo bar
>
> Without the --pipe:
>
> #!/usr/bin/parallel --shebang-program -k -j24 /usr/bin/python
>
> should work like:
>
> parallel -k -j24 /usr/bin/python my_program {}
>
> So:
>
> my_program foo bar
> (echo foo; echo bar) | my_program
>
> should do the same as:
>
> parallel -k -j24 /usr/bin/python my_program {} ::: foo bar
These all work now (--shebang-wrap instead of --shebang-program). But
please test them. Also if you come up with cool examples please post them.
> We should allow for putting options on the command interpreter. E.g:
>
> #!/usr/bin/parallel --shebang-program -k -j24 /usr/bin/perl -p
>
> Also without the --pipe the {} should work as expected:
>
> #!/usr/bin/parallel --shebang-program -k -j24 /usr/bin/perl -p {}
> {.}.out > {.}.log
This, however, does not work. The above should run:
/usr/bin/parallel -k -j24 /usr/bin/perl -p my_program "{} {.}.out > {.}.log"
The problem is that the script file name needs to be put into the
middle of the command line (above it is put after -p). As I do not see
a way to find right place to put in the script file name, I do not see
a way to get that part working.
/Ole