Bash script does not continue to read the next line of file

Question

I have a shell script that saves the output of a command that is executed to a CSV file. It reads the command it has to execute from a shell script which is in this format:

Answer 1

I would add echos before and after the eval to see what it's about to eval (in case it's treating the whole file as one big long line) and after (in case one of the ffmpeg commands is taking forever).

Answer 2

Unless you are planning to read something from standard input after the loop, you don't need to preserve and restore the original standard input (though it is good to see you know how).

Similarly, I don't see a reason for dinking with IFS at all. There is certainly no need to restore the value of IFS before exit - this is a real shell you are using, not a DOS BAT file.

When you do:

read var1 var2 var3

the shell assigns the first field to $var1, the second to $var2, and the rest of the line to $var3. In the case where there's just one variable - your script, for example - the whole line goes into the variable, just as you want it to.

Inside the process line function, you probably don't want to throw away error output from the executed command. You probably do want to think about checking the exit status of the command. The echo with error redirection is ... unusual, and overkill. If you're sufficiently sure that the commands can't fail, then go ahead with ignoring the error. Is the command 'chatty'; if so, throw away the chat by all means. If not, maybe you don't need to throw away standard output, either.

The script as a whole should probably diagnose when it is given multiple files to process since it ignores the extraneous ones.

You could simplify your file handling by using just:

cat "$@" |
while read line
do
    processline "$line"
done

The cat command automatically reports errors (and continues after them) and processes all the input files, or reads standard input if there are no arguments left. The use of double quotes around the variable means that it is passed as a single unit (and therefore unparsed into separate words).

The use of date and bc is interesting - I'd not seen that before.

All in all, I'd be looking at something like:

#!/bin/bash
# Time execution of commands read from a file, line by line.
# Log commands and times to CSV logfile "file.csv"

processLine(){
    START=$(date +%s.%N)
    eval "$@" > /dev/null
    STATUS=$?
    END=$(date +%s.%N)
    DIFF=$(echo "$END - $START" | bc)
    echo "$line, $START, $END, $DIFF, $STATUS" >> file.csv
    echo "${DIFF}s: $STATUS: $line"
}

cat "$@" |
while read line
do
    processLine "$line"
done

Answer 3

ffmpeg reads STDIN and exhausts it. The solution is to call ffmpeg with:

 ffmpeg </dev/null ...

See the detailed explanation here: http://mywiki.wooledge.org/BashFAQ/089

Answer 4

I just had the same problem.

I believe ffmpeg is responsible for this behaviour.

My solution for this problem:

1) Call ffmpeg with an "&" at the end of your ffmpeg command line

2) Since now the skript will not wait till completion of the ffmpeg process, we have to prevent our script from starting several ffmpeg processes. We achieve this goal by delaying the loop pass while there is at least one running ffmpeg process.

#!/bin/bash

cat FileList.txt |
while read VideoFile; do
    <place your ffmpeg command line here> &
    FFMPEGStillRunning="true"
    while [ "$FFMPEGStillRunning" = "true" ]; do
        Process=$(ps -C ffmpeg | grep -o -e "ffmpeg" )
        if [ -n "$Process" ]; then
            FFMPEGStillRunning="true"
        else
            FFMPEGStillRunning="false"
        fi 
        sleep 2s
    done
done

Answer 5

I think that should do the same and seems to be correct:

#!/bin/bash

CSVFILE=/tmp/file.csv

cat "$@" | while read line; do
    echo "Executing '$line'"
    START=$(date +%s)
    eval $line &> /dev/null
    END=$(date +%s)
    let DIFF=$END-$START

    echo "$line, $START, $END, $DIFF" >> "$CSVFILE"
    echo "It took ${DIFF}s"
done

no?