When to use xargs when piping?

Question

I am new to bash and I am trying to understand the use of xargs, which is still not clear for me. For example:

history | grep ls

H

Answer 1

GNU Parallel can do the same as xargs, but does not have the broken and exploitable "features".

You can learn GNU Parallel by looking at examples http://www.gnu.org/software/parallel/man.html#EXAMPLE:-Working-as-xargs--n1.-Argument-appending and walking through the tutorial http://www.gnu.org/software/parallel/parallel_tutorial.html

Answer 2

xargs(1) is dangerous (broken, exploitable, etc.) when reading non-NUL-delimited input.

If you're working with filenames, use find's -exec [command] {} + instead. If you can get NUL-delimited output, use xargs -0.

Answer 3

Short answer: Avoid xargs for now. Return to xargs when you have written dozens or hundreds of scripts.

Commands can get their input from parameters (like rm bad_example) or can get the input from stdin (not just the y on the question after rm -i is_this_bad_too, but also read answer). Other commands like grep and sed will look for parameters and when the parameters don't show the input, switch to the input.
Your grep example works fine reading from stdin, nothing special needed.
Your ls needs the output of find as a parameter. xargs is just one way to turn things around. Use man xargs for more about xargs. Alternatives:

find /etc -name "*.txt" -exec ls -l {} \;
find /etc -name "*.txt" -ls
ls -l $(find /etc -name "*.txt" )
ls /etc/*.txt

First try to see which of this commands is best when you have a nasty filename with spaces.txt in /etc.

Answer 4

When you use piping without xargs, the actual data is fed into the next command. On the other hand, when using piping with xargs, the actual data is viewed as a parameter to the next command. To give a concrete example, say you have a folder with a.txt and b.txt. a.txt contains just a single line 'hello world!', and b.txt is just empty.

If you do

ls | grep txt

you would end up getting the output:

a.txt
b.txt

Yet, if you do

ls | xargs grep txt

you would get nothing since neither file a.txt nor b.txt contains the word txt.

If the command is

ls | xargs grep hello

you would get:

hello world!

That's because with xargs, the two filenames given by ls are passed to grep as arguments, rather than the actual content.

Answer 5

To answer your question, xargs can be used when you need to take the output from one command and use it as an argument to another. In your first example, grep takes the data from standard input, rather than as an argument. So, xargs is not needed.

xargs takes data from standard input and executes a command. By default, the data is appended to the end of the command as an argument. It can be inserted anywhere however, using a placeholder for the input. The traditional placeholder is {}; using that, your example command might then be written as:

find /etc -name "*.txt" | xargs -I {} ls -l {}

If you have 3 text files in /etc you'll get a full directory listing of each. Of course, you could just have easily written ls -l /etc/*.txt and saved the trouble.

Another example lets you rename those files, and requires the placeholder {} to be used twice.

find /etc -name "*.txt" | xargs -I {} mv {} {}.bak

These are both bad examples, and will break as soon as you have a filename containing whitespace. You can work around that by telling find to separate filenames with a null character.

find /etc -print0 -name "*.txt" | xargs -I {} -0 mv {} {}.bak

My personal opinion is that there are almost always alternatives to using xargs, and you will be better served by learning those.