Looping through the content of a file in Bash

Question

How do I iterate through each line of a text file with Bash?

With this script:

echo \"Start!\"
for p in (peptides.txt)
do
    echo \"${p}\"
done
         


        

Answer 1

A few more things not covered by other answers:

Reading from a delimited file

# ':' is the delimiter here, and there are three fields on each line in the file
# IFS set below is restricted to the context of `read`, it doesn't affect any other code
while IFS=: read -r field1 field2 field3; do
  # process the fields
  # if the line has less than three fields, the missing fields will be set to an empty string
  # if the line has more than three fields, `field3` will get all the values, including the third field plus the delimiter(s)
done < input.txt

Reading from the output of another command, using process substitution

while read -r line; do
  # process the line
done < <(command ...)

This approach is better than command ... | while read -r line; do ... because the while loop here runs in the current shell rather than a subshell as in the case of the latter. See the related post A variable modified inside a while loop is not remembered.

Reading from a null delimited input, for example find ... -print0

while read -r -d '' line; do
  # logic
  # use a second 'read ... <<< "$line"' if we need to tokenize the line
done < <(find /path/to/dir -print0)

Related read: BashFAQ/020 - How can I find and safely handle file names containing newlines, spaces or both?

Reading from more than one file at a time

while read -u 3 -r line1 && read -u 4 -r line2; do
  # process the lines
  # note that the loop will end when we reach EOF on either of the files, because of the `&&`
done 3< input1.txt 4< input2.txt

Based on @chepner's answer here:

-u is a bash extension. For POSIX compatibility, each call would look something like read -r X <&3.

Reading a whole file into an array (Bash versions earlier to 4)

while read -r line; do
    my_array+=("$line")
done < my_file

If the file ends with an incomplete line (newline missing at the end), then:

while read -r line || [[ $line ]]; do
    my_array+=("$line")
done < my_file

Reading a whole file into an array (Bash versions 4x and later)

readarray -t my_array < my_file

or

mapfile -t my_array < my_file

And then

for line in "${my_array[@]}"; do
  # process the lines
done

• More about the shell builtins read and readarray commands - GNU

• More about IFS - Wikipedia

• BashFAQ/001 - How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?

Related posts:

• Creating an array from a text file in Bash
• What is the difference between thee approaches to reading a file that has just one line?
• Bash while read loop extremely slow compared to cat, why?