Splitting command line args with GNU parallel

问题

Using GNU parallel: http://www.gnu.org/software/parallel/

I have a program that takes two arguments, e.g.

$ ./prog file1 file2
$ ./prog file2 file3
...
$ ./prog file23456 file23457

I'm using a script that generates the file name pairs, however this poses a problem because the result of the script is a single string - not a pair. like:

$ ./prog "file1 file2"

GNU parallel seems to have a slew of tricks up its sleeves, I wonder if there's one for splitting text around separators:

$ generate_file_pairs | parallel ./prog ?  
  # where ? is text under consideration, like "file1 file2"

The easy work around is to split the args manually in prog, but I'd like to know if it's possible in GNU parallel.

回答1:

You are probably looking for --colsep.

generate_file_pairs | parallel --colsep ' ' ./prog {1} {2}

Read man parallel for more. And watch the intro video if you have not already done so http://www.youtube.com/watch?v=OpaiGYxkSuQ

回答2:

You are looking for -n option of parallel. This is what you are looking for:

./generate_file_pairs | parallel -n 2 ./prog {}

Excerpt from GNU Parallel Doc:

-n max-args
    Use at most max-args arguments per command line. Fewer than max-args 
    arguments will be used if the size (see the -s option) is exceeded, 
    unless the -x option is given, in which case GNU parallel will exit.

回答3:

In Parallel's manual, it is said:

If no command is given, the line of input is executed ... GNU parallel can often be used as a substitute for xargs or cat | bash.

So take a try of:

generate command | parallel

Try to understand the output of this:

for i in {1..5};do echo "echo $i";done | parallel

来源：https://stackoverflow.com/questions/6255286/splitting-command-line-args-with-gnu-parallel

标签

bash

file-processing

gnu-parallel