xargs with multiple arguments

Question

I have a source input, input.txt

a.txt
b.txt
c.txt

I want to feed these input into a program as the following:

<         


        

Answer 1

None of the solutions given so far deals correctly with file names containing space. Some even fail if the file names contain ' or ". If your input files are generated by users, you should be prepared for surprising file names.

GNU Parallel deals nicely with these file names and gives you (at least) 3 different solutions. If your program takes 3 and only 3 arguments then this will work:

(echo a1.txt; echo b1.txt; echo c1.txt;
 echo a2.txt; echo b2.txt; echo c2.txt;) |
parallel -N 3 my-program --file={1} --file={2} --file={3}

Or:

(echo a1.txt; echo b1.txt; echo c1.txt;
 echo a2.txt; echo b2.txt; echo c2.txt;) |
parallel -X -N 3 my-program --file={}

If, however, your program takes as many arguments as will fit on the command line:

(echo a1.txt; echo b1.txt; echo c1.txt;
 echo d1.txt; echo e1.txt; echo f1.txt;) |
parallel -X my-program --file={}

Watch the intro video to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ

Answer 2

You can use sed to prefix --file= to each line and then call xargs:

sed -e 's/^/--file=/' input.txt | xargs my-program

Answer 3

I stumbled on a similar problem and found a solution which I think is nicer and cleaner than those presented so far.

The syntax for xargs that I have ended with would be (for your example):

xargs -I X echo --file=X

with a full command line being:

my-program $(cat input.txt | xargs -I X echo --file=X)

which will work as if

my-program --file=a.txt --file=b.txt --file=c.txt

was done (providing input.txt contains data from your example).

---

Actually, in my case I needed to first find the files and also needed them sorted so my command line looks like this:

my-program $(find base/path -name "some*pattern" -print0 | sort -z | xargs -0 -I X echo --files=X)

Few details that might not be clear (they were not for me):

• some*pattern must be quoted since otherwise shell would expand it before passing to find.
• -print0, then -z and finally -0 use null-separation to ensure proper handling of files with spaces or other wired names.

Note however that I didn't test it deeply yet. Though it seems to be working.

Answer 4

I was looking for a solution for this exact problem and came to the conclution of coding a script in the midle.

to transform the standard output for the next example use the -n '\n' delimeter

example:

 user@mybox:~$ echo "file1.txt file2.txt" | xargs -n1 ScriptInTheMiddle.sh

 inside the ScriptInTheMidle.sh:
 !#/bin/bash
 var1=`echo $1 | cut -d ' ' -f1 `
 var2=`echo $1 | cut -d ' ' -f2 `
 myprogram  "--file1="$var1 "--file2="$var2 

For this solution to work you need to have a space between those arguments file1.txt and file2.txt, or whatever delimeter you choose, one more thing, inside the script make sure you check -f1 and -f2 as they mean "take the first word and take the second word" depending on the first delimeter's position found (delimeters could be ' ' ';' '.' whatever you wish between single quotes . Add as many parameters as you wish.

Problem solved using xargs, cut , and some bash scripting.

Cheers!

if you wanna pass by I have some useful tips http://hongouru.blogspot.com

Answer 5

xargs doesn't work that way. Try:

  myprogram $(sed -e 's/^/--file=/' input.txt)

Answer 6

Nobody has mentioned echoing out from a loop yet, so I'll put that in for completeness sake (it would be my second approach, the sed one being the first):

for line in $(< input.txt) ; do echo --file=$line ; done | xargs echo my-program