Iterate over a list of files with spaces

Question

I want to iterate over a list of files. This list is the result of a find command, so I came up with:

getlist() {
  for f in $(find . -iname \"f         


        

Answer 1

find . -iname "foo*" -print0 | xargs -L1 -0 echo "File found:"

Answer 2

Since you aren't doing any other type of filtering with find, you can use the following as of bash 4.0:

shopt -s globstar
getlist() {
    for f in **/foo*
    do
        echo "File found: $f"
        # do something useful
    done
}

The **/ will match zero or more directories, so the full pattern will match foo* in the current directory or any subdirectories.

Answer 3

There is also a very simple solution: rely on bash globbing

$ mkdir test
$ cd test
$ touch "stupid file1"
$ touch "stupid file2"
$ touch "stupid   file 3"
$ ls
stupid   file 3  stupid file1     stupid file2
$ for file in *; do echo "file: '${file}'"; done
file: 'stupid   file 3'
file: 'stupid file1'
file: 'stupid file2'

Note that I am not sure this behavior is the default one but I don't see any special setting in my shopt so I would go and say that it should be "safe" (tested on osx and ubuntu).

Answer 4

Ok - my first post on Stack Overflow!

Though my problems with this have always been in csh not bash the solution I present will, I'm sure, work in both. The issue is with the shell's interpretation of the "ls" returns. We can remove "ls" from the problem by simply using the shell expansion of the * wildcard - but this gives a "no match" error if there are no files in the current (or specified folder) - to get around this we simply extend the expansion to include dot-files thus: * .* - this will always yield results since the files . and .. will always be present. So in csh we can use this construct ...

foreach file (* .*)
   echo $file
end

if you want to filter out the standard dot-files then that is easy enough ...

foreach file (* .*)
   if ("$file" == .) continue
   if ("file" == ..) continue
   echo $file
end

The code in the first post on this thread would be written thus:-

getlist() {
  for f in $(* .*)
  do
    echo "File found: $f"
    # do something useful
  done
}

Hope this helps!

Answer 5

Another solution for job...

Goal was :

• select/filter filenames recursively in directories
• handle each names (whatever space in path...)

#!/bin/bash  -e
## @Trick in order handle File with space in their path...
OLD_IFS=${IFS}
IFS=$'\n'
files=($(find ${INPUT_DIR} -type f -name "*.md"))
for filename in ${files[*]}
do
      # do your stuff
      #  ....
done
IFS=${OLD_IFS}

Answer 6

There are several workable ways to accomplish this.

If you wanted to stick closely to your original version it could be done this way:

getlist() {
        IFS=$'\n'
        for file in $(find . -iname 'foo*') ; do
                printf 'File found: %s\n' "$file"
        done
}

This will still fail if file names have literal newlines in them, but spaces will not break it.

However, messing with IFS isn't necessary. Here's my preferred way to do this:

getlist() {
    while IFS= read -d $'\0' -r file ; do
            printf 'File found: %s\n' "$file"
    done < <(find . -iname 'foo*' -print0)
}

If you find the < <(command) syntax unfamiliar you should read about process substitution. The advantage of this over for file in $(find ...) is that files with spaces, newlines and other characters are correctly handled. This works because find with -print0 will use a null (aka \0) as the terminator for each file name and, unlike newline, null is not a legal character in a file name.

The advantage to this over the nearly-equivalent version

getlist() {
        find . -iname 'foo*' -print0 | while read -d $'\0' -r file ; do
                printf 'File found: %s\n' "$file"
        done
}

Is that any variable assignment in the body of the while loop is preserved. That is, if you pipe to while as above then the body of the while is in a subshell which may not be what you want.

The advantage of the process substitution version over find ... -print0 | xargs -0 is minimal: The xargs version is fine if all you need is to print a line or perform a single operation on the file, but if you need to perform multiple steps the loop version is easier.

EDIT: Here's a nice test script so you can get an idea of the difference between different attempts at solving this problem

#!/usr/bin/env bash

dir=/tmp/getlist.test/
mkdir -p "$dir"
cd "$dir"

touch       'file not starting foo' foo foobar barfoo 'foo with spaces'\
    'foo with'$'\n'newline 'foo with trailing whitespace      '

# while with process substitution, null terminated, empty IFS
getlist0() {
    while IFS= read -d $'\0' -r file ; do
            printf 'File found: '"'%s'"'\n' "$file"
    done < <(find . -iname 'foo*' -print0)
}

# while with process substitution, null terminated, default IFS
getlist1() {
    while read -d $'\0' -r file ; do
            printf 'File found: '"'%s'"'\n' "$file"
    done < <(find . -iname 'foo*' -print0)
}

# pipe to while, newline terminated
getlist2() {
    find . -iname 'foo*' | while read -r file ; do
            printf 'File found: '"'%s'"'\n' "$file"
    done
}

# pipe to while, null terminated
getlist3() {
    find . -iname 'foo*' -print0 | while read -d $'\0' -r file ; do
            printf 'File found: '"'%s'"'\n' "$file"
    done
}

# for loop over subshell results, newline terminated, default IFS
getlist4() {
    for file in "$(find . -iname 'foo*')" ; do
            printf 'File found: '"'%s'"'\n' "$file"
    done
}

# for loop over subshell results, newline terminated, newline IFS
getlist5() {
    IFS=$'\n'
    for file in $(find . -iname 'foo*') ; do
            printf 'File found: '"'%s'"'\n' "$file"
    done
}


# see how they run
for n in {0..5} ; do
    printf '\n\ngetlist%d:\n' $n
    eval getlist$n
done

rm -rf "$dir"