Bash: read a file line-by-line and process each segment as parameters to other prog

问题

I have some dirty work to do, so a Bash script seems to be a good choice. I'm new to Bash, and the experience makes me kind of frustrated.

The file mapfiles.txt consists of lines as follow. Each line has four segments separated by a white space. Each segment represents a input parameter to an external program name 'prog'. For example, "cm19_1.png" is the filename, "0001" the index, "121422481" the longitude, and "31035995" the latitude.

File: mapfiles.txt

cm19_1.png 0001 121422481 31035995
cm19_2.png 0002 121423224 31035995
cm19_3.png 0003 121423967 31035995
…

I want to execute similar commands to each line. As show below, the prog's input parameter order is slightly different. So it makes sense to write a bash script to handle the repeated work.

[Usage] prog <index> <longitude> <latitude> <filename>
example: prog 0001 121422481 31035995 cm19_1.png

Generally, the bash script will operate in this way:

Read one line from mapfiles.txt
Split the segments
Call the prog with a correct parameter order

Here comes run.sh.

#!/bin/sh

input=mapfiles.txt
cmd=prog

while read line
do
        file=$(echo $line | cut -d' ' -f1)
        key=$(echo $line | cut -d' ' -f2)
        log=$(echo $line | cut -d' ' -f3)
        lat=$(echo $line | cut -d' ' -f4)
        echo $cmd $key $log $lat $file
done < "$input"

What I expected is

prog 0001 121422481 31035995 cm19_1.png
prog 0002 121423224 31035995 cm19_2.png
prog 0003 121423967 31035995 cm19_3.png
…

The ACTUAL result I got is

 cm19_1.png21422481 31035995
 cm19_2.png21423224 31035995
 cm19_3.png21423967 31035995

Problems that confused me

Where is 'prog'?
Where is the white space?
What's wrong with the parameter order?

Hmm… I wrote this script on my Mac using vim and copy it to a Scientific Linux box and a gentoo box. These three guys get the same ridiculous outputs.

回答1:

You can simplify this a lot:

while read file key log lat
do
  echo "$cmd" "$key" "$log" "$lat" "$file"
done < "$input"

回答2:

Using GNU Parallel you can do it in a single line + you get it done in parallel for free:

cat mapfile.txt | parallel --colsep '\s' prog {2} {3} {4} {1}

Watch the intro videos to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ

回答3:

prog could have disappeared because $cmd is not exported. Your version of /bin/sh might execute the while statement in a separate shell. This should not be the case, and this is not the case for my installation of bash, but perhaps yours behaves in interesting ways in this department.

UPD I see that you have several boxes that give the same results. This makes the subshell theory unlikely. Perhaps you have some funny characters in your script and/or source file.

I have copied and pasted your script and your source file to my gentoo box and it gives the expected results. Perhaps you should do the same, and compare the files with your original ones.

来源：https://stackoverflow.com/questions/7619438/bash-read-a-file-line-by-line-and-process-each-segment-as-parameters-to-other-p

标签

bash

parameters

line-by-line