Make xargs execute the command once for each line of input

Question

How can I make xargs execute the command exactly once for each line of input given? It\'s default behavior is to chunk the lines and execute the command once, passing multip

Answer 1

The following command will find all the files (-type f) in /path and then copy them using cp to the current folder. Note the use if -I % to specify a placeholder character in the cp command line so that arguments can be placed after the file name.

find /path -type f -print0 | xargs -0 -I % cp % .

Tested with xargs (GNU findutils) 4.4.0

Answer 2

How can I make xargs execute the command exactly once for each line of input given?

-L 1 is the simple solution but it does not work if any of the files contain spaces in them. This is a key function of find's -print0 argument – to separate the arguments by '\0' character instead of whitespace. Here's an example:

echo "file with space.txt" | xargs -L 1 ls
ls: file: No such file or directory
ls: with: No such file or directory
ls: space.txt: No such file or directory

A better solution is to use tr to convert newlines to null (\0) characters, and then use the xargs -0 argument. Here's an example:

echo "file with space.txt" | tr '\n' '\0' | xargs -0 ls
file with space.txt

If you then need to limit the number of calls you can use the -n 1 argument to make one call to the program for each input:

echo "file with space.txt" | tr '\n' '\0' | xargs -0 -n 1 ls

This also allows you to filter the output of find before converting the breaks into nulls.

find . -name \*.xml | grep -v /target/ | tr '\n' '\0' | xargs -0 tar -cf xml.tar

Answer 3

find path -type f | xargs -L1 command 

is all you need.

Answer 4

It seems to me all existing answers on this page are wrong, including the one marked as correct. That stems from the fact that the question is ambiguously worded.

Summary:   If you want to execute the command "exactly once for each line of input," passing the entire line (without newline) to the command as a single argument, then this is the best UNIX-compatible way to do it:

... | tr '\n' '\0' | xargs -0 -n1 ...

If you are using GNU xargs and don't need to be compatible with all other UNIX's (FreeBSD, Mac OS X, etc.) then you can use the GNU-specific option -d:

... | xargs -d\\n -n1 ...

Now for the long explanation…

---

There are two issues to take into account when using xargs:

1. how does it split the input into "arguments"; and
2. how many arguments to pass the child command at a time.

To test xargs' behavior, we need an utility that shows how many times it's being executed and with how many arguments. I don't know if there is a standard utility to do that, but we can code it quite easily in bash:

#!/bin/bash
echo -n "-> "; for a in "$@"; do echo -n "\"$a\" "; done; echo

Assuming you save it as show in your current directory and make it executable, here is how it works:

$ ./show one two 'three and four'
-> "one" "two" "three and four" 

Now, if the original question is really about point 2. above (as I think it is, after reading it a few times over) and it is to be read like this (changes in bold):

How can I make xargs execute the command exactly once for each argument of input given? Its default behavior is to chunk the input into arguments and execute the command as few times as possible, passing multiple arguments to each instance.

then the answer is -n 1.

Let's compare xargs' default behavior, which splits the input around whitespace and calls the command as few times as possible:

$ echo one two 'three and four' | xargs ./show 
-> "one" "two" "three" "and" "four" 

and its behavior with -n 1:

$ echo one two 'three and four' | xargs -n 1 ./show 
-> "one" 
-> "two" 
-> "three" 
-> "and" 
-> "four" 

If, on the other hand, the original question was about point 1. input splitting and it was to be read like this (many people coming here seem to think that's the case, or are confusing the two issues):

How can I make xargs execute the command with exactly one argument for each line of input given? Its default behavior is to chunk the lines around whitespace.

then the answer is more subtle.

One would think that -L 1 could be of help, but it turns out it doesn't change argument parsing. It only executes the command once for each input line, with as many arguments as were there on that input line:

$ echo $'one\ntwo\nthree and four' | xargs -L 1 ./show 
-> "one" 
-> "two" 
-> "three" "and" "four" 

Not only that, but if a line ends with whitespace, it is appended to the next:

$ echo $'one \ntwo\nthree and four' | xargs -L 1 ./show 
-> "one" "two" 
-> "three" "and" "four" 

Clearly, -L is not about changing the way xargs splits the input into arguments.

The only argument that does so in a cross-platform fashion (excluding GNU extensions) is -0, which splits the input around NUL bytes.

Then, it's just a matter of translating newlines to NUL with the help of tr:

$ echo $'one \ntwo\nthree and four' | tr '\n' '\0' | xargs -0 ./show 
-> "one " "two" "three and four" 

Now the argument parsing looks all right, including the trailing whitespace.

Finally, if you combine this technique with -n 1, you get exactly one command execution per input line, whatever input you have, which may be yet another way to look at the original question (possibly the most intuitive, given the title):

$ echo $'one \ntwo\nthree and four' | tr '\n' '\0' | xargs -0 -n1 ./show 
-> "one " 
-> "two" 
-> "three and four" 

As mentioned above, if you are using GNU xargs you can replace the tr with the GNU-specific option -d:

$ echo $'one \ntwo\nthree and four' | xargs -d\\n -n1 ./show 
-> "one " 
-> "two" 
-> "three and four" 

Answer 5

These two ways also work, and will work for other commands that are not using find!

xargs -I '{}' rm '{}'
xargs -i rm '{}'

example use case:

find . -name "*.pyc" | xargs -i rm '{}'

will delete all pyc files under this directory even if the pyc files contain spaces.

Answer 6

You can limit the number of lines, or arguments (if there are spaces between each argument) using the --max-lines or --max-args flags, respectively.

  -L max-lines
         Use at most max-lines nonblank input lines per command line.  Trailing blanks cause an input line to be logically continued on the next  input
         line.  Implies -x.

  --max-lines[=max-lines], -l[max-lines]
         Synonym  for  the -L option.  Unlike -L, the max-lines argument is optional.  If max-args is not specified, it defaults to one.  The -l option
         is deprecated since the POSIX standard specifies -L instead.

  --max-args=max-args, -n max-args
         Use at most max-args arguments per command line.  Fewer than max-args arguments will be used if the size (see  the  -s  option)  is  exceeded,
         unless the -x option is given, in which case xargs will exit.