How to create an array from the lines of a command's output

Question

I have a file called failedfiles.txt with the following content:

failed1
failed2
failed3

I need to use grep to return the content on each line in that file, and save the output in a list to be accessed. So I want something like this:

temp_list=$(grep "[a-z]" failedfiles.txt)

However, the problem with this is that when I type

echo ${temp_list[0]}

I get the following output:

failed1 failed2 failed3

But what I want is when I do:

echo ${temp_list[0]}

to print

failed1

and when I do:

echo ${temp_list[1]}

to print

failed2

Thanks.

Answer 1

You did not create an array. What you did was Command Substitution which would simply put the output of a command into a variable.

In order to create an array, say:

temp_list=( $(grep "[a-z]" failedfiles.txt) )

You might also want to refer to Guide on Arrays.

Answer 2

@devnull's helpful answer explains why your code didn't work as expected: command substitution always returns a single string (possibly composed of multiple lines).

However, simply putting (...) around a command substitution to create an array of lines will only work as expected if the lines output by the command do not have embedded spaces - otherwise, each individual (whitespace-separated) word will become its own array element.

---

Capturing command output lines at once, in an array:

To capture the lines output by an arbitrary command in an array, use the following:

• bash < 4 (e.g., on OSX as of OS X 10.9.2): use read -a

IFS=$'\n' read -rd '' -a linesArray <<<"$(grep "[a-z]" failedfiles.txt)"

• bash >= 4: use readarray:

readarray -t linesArray <<<"$(grep "[a-z]" failedfiles.txt)"

Note:

• <<< initiates a so-called here-string, which pipes the string to its right (which happens to be the result of a command substitution here) into the command on the left via stdin.
  • While command <<< string is functionally equivalent to echo string | command in principle, the crucial difference is that the latter creates subshells, which make variable assignments in command pointless - they are localized to each subshell.
• An alternative to combining here-strings with command substitution is [input] process substitution - <(...) - which, simply put, allows using a command's output as if it were an input file; the equivalent of <<<"$(command)" is < <(command).
• read: -a reads into an array, and IFS=$'\n' ensures that every line is considered a separate field and thus read into its own array element; -d '' ensures that ALL lines are read at once (before breaking them into fields); -r turns interpretation of escape sequence in the input off.
• readarray (also callable as mapfile) directly breaks input lines into an array of lines; -t ensures that the terminating \n is NOT included in the array elements.

---

Looping over command output lines:

If there is no need to capture all lines in an array at once and looping over a command's output line by line is sufficient, use the following:

while IFS= read -r line; do
  # ...
done < <(grep "[a-z]" failedfiles.txt)

• IFS= ensures that each line is read unmodified in terms of whitespace; remove it to have leading and trailing whitespace trimmed.
• -r ensures that the lines are read 'raw' in that substrings in the input that look like escape sequences - e.g., \t - are NOT interpreted as such.
• Note the use of [input] process substitution (explained above) to provide the command output as input to the read loop.

Answer 3

The proper and portable way to loop over lines in a file is simply

while read -r line; do
    ... something with "$line"
done <failedfiles.txt