Shell pipe: Exit immediately when one command fails

Question

I\'m using a pipe of several commands in bash. Is there a way of configuring bash to terminate all commands in the whole pipeline immediately should one of the commands fail

Answer 1

The bash documentation says in its section about pipelines:

Each command in a pipeline is executed in its own subshell [...]

"In its own subshell" means that a new bash process is spawned, which then gets to execute the actual command. Each subshell starts successfully, even when it immediately determines that the command it is asked to execute doesn't exist.

This explains why the entire pipe can be set up successfully even when one of the commands is nonsense. Bash does not check if each command can be run, it delegates that to the subshells. That also explains why, for example, the command nonexisting-command | touch hello will throw a "command not found" error, but the file hello will be created nonetheless.

In the same section, it also says:

The shell waits for all commands in the pipeline to terminate before returning a value.

In sleep 5 | nonexisting-command, as A.H. pointed out, the sleep 5 terminates after 5 seconds, not immediately, hence the shell will also wait 5 seconds.

I don't know why the implementation was done this way. In cases like yours, the behavior is surely not as one would expect.

Anyway, one slightly ugly workaround is to use FIFOs:

mkfifo myfifo
./long-running-script.sh > myfifo &
whoops-a-typo < myfifo

Here, the long-running-script.sh is started and then the scripts fails immediately on the next line. Using mutiple FIFOs, this could be extended to pipes with more than two commands.

Answer 2

sleep 5 doesn't produce any output until it finishes, while find / immediately produces output that bash attempts to pipe to false.

Answer 3

The first program does not know whether the second is terminated or not until it tries to write some date into the pipe. In case the second is terminated, the first receives the SIGPIPE which usually causes immediate exit.

You can force the first line of output to be piped immediately after staring, like this:

(sleep 0.1; echo; command1) | command2

This 100ms sleep is intended to wait until possible command2 exit right after starting. Of course, if command2 exits after 2 seconds, and command1 will be silent for 60 seconds, the whole shell command will return only after 60.1 seconds.

Answer 4

find / |false fails faster because the first write(2) system call from find fails with the error EPIPE (Broken pipe). This is because false has been already terminated and hence the pipe between these two commands has been closed already on one side.

If find would ignore that error (it could do so in theory) it would by also "fail slow".

(sleep 5 && echo "Hello") | false is "fail slow", because the first part, sleep, does not "test" the pipe by writing to it. After 5 seconds the echo also get the EPIPE error. Whether this error terminates the first part in this case or not is not important to the question.

Answer 5

The following code seems to work by Dash, but EXIT traps within pipe do not work in Bash; maybe it is a bug in Bash.

#!/bin/sh

echo PID of the shell: $$

trap 'echo In INT trap >&2; trap - EXIT INT; kill -s INT $$' INT

(
    # now in subshell
    pidofsubshell=$(exec sh -c 'echo "$PPID"')
    # $BASHPID can be used as a value, when using Bash
    echo PID of subshell: $pidofsubshell

    fifo=$(mktemp -u); shells=$(mktemp) childs=$(mktemp)
    mkfifo $fifo
    trap 'echo In sub trap >&2; rm $fifo $shells $childs; trap - EXIT; exit' EXIT HUP TERM INT ALRM

    pipe_trap() {
        code=$?
        echo In sub sub trap $1 >&2
        echo $1 $code >> $fifo
    }
    { trap 'echo In pipe signal trap >&2; kill $(cat $childs $shells) 2>/dev/null' INT HUP TERM ALRM
        { trap 'pipe_trap 1' EXIT
            sleep 30; } \
        | { trap 'pipe_trap 2' EXIT
            sleep 50 & sleep 2; } \
        | { trap 'pipe_trap 3' EXIT
            sleep 40; } &
    }

    echo ps tail:
    ps xao pid,ppid,pgid,sid,command | head -n 1
    ps xao pid,ppid,pgid,sid,command | tail -n 15
        ps -o pid= --ppid $pidofsubshell | head -n -2 > $shells # strip pids of ps and head
    echo shells:
    cat $shells
        while read -r ppid; do ps -o pid= --ppid $ppid; done <$shells >$childs
    echo childs of above
    cat $childs

    { 
        IFS=' ' read -r id exitcode
        echo Pipe part nr. $id terminated first with code $exitcode\; killing the remaining processes.
        kill $(cat $childs $shells) 2>/dev/null
    } < $fifo
)

echo
echo After subshell:
ps xao pid,ppid,pgid,sid,command | head -n 1
ps xao pid,ppid,pgid,sid,command | tail -n 15