Pipe output and capture exit status in Bash

Question

I want to execute a long running command in Bash, and both capture its exit status, and tee its output.

So I do this:

command | tee out.txt
ST=$?


        

Answer 1

The simplest way to do this in plain bash is to use process substitution instead of a pipeline. There are several differences, but they probably don't matter very much for your use case:

• When running a pipeline, bash waits until all processes complete.
• Sending Ctrl-C to bash makes it kill all the processes of a pipeline, not just the main one.
• The pipefail option and the PIPESTATUS variable are irrelevant to process substitution.
• Possibly more

With process substitution, bash just starts the process and forgets about it, it's not even visible in jobs.

Mentioned differences aside, consumer < <(producer) and producer | consumer are essentially equivalent.

If you want to flip which one is the "main" process, you just flip the commands and the direction of the substitution to producer > >(consumer). In your case:

command > >(tee out.txt)

Example:

$ { echo "hello world"; false; } > >(tee out.txt)
hello world
$ echo $?
1
$ cat out.txt
hello world

$ echo "hello world" > >(tee out.txt)
hello world
$ echo $?
0
$ cat out.txt
hello world

As I said, there are differences from the pipe expression. The process may never stop running, unless it is sensitive to the pipe closing. In particular, it may keep writing things to your stdout, which may be confusing.

Answer 2

By combining PIPESTATUS[0] and the result of executing the exit command in a subshell, you can directly access the return value of your initial command:

command | tee ; ( exit ${PIPESTATUS[0]} )

Here's an example:

# the "false" shell built-in command returns 1
false | tee ; ( exit ${PIPESTATUS[0]} )
echo "return value: $?"

will give you:

return value: 1

Answer 3

Pure shell solution:

% rm -f error.flag; echo hello world \
| (cat || echo "First command failed: $?" >> error.flag) \
| (cat || echo "Second command failed: $?" >> error.flag) \
| (cat || echo "Third command failed: $?" >> error.flag) \
; test -s error.flag  && (echo Some command failed: ; cat error.flag)
hello world

And now with the second cat replaced by false:

% rm -f error.flag; echo hello world \
| (cat || echo "First command failed: $?" >> error.flag) \
| (false || echo "Second command failed: $?" >> error.flag) \
| (cat || echo "Third command failed: $?" >> error.flag) \
; test -s error.flag  && (echo Some command failed: ; cat error.flag)
Some command failed:
Second command failed: 1
First command failed: 141

Please note the first cat fails as well, because it's stdout gets closed on it. The order of the failed commands in the log is correct in this example, but don't rely on it.

This method allows for capturing stdout and stderr for the individual commands so you can then dump that as well into a log file if an error occurs, or just delete it if no error (like the output of dd).

Answer 4

Base on @brian-s-wilson 's answer; this bash helper function:

pipestatus() {
  local S=("${PIPESTATUS[@]}")

  if test -n "$*"
  then test "$*" = "${S[*]}"
  else ! [[ "${S[@]}" =~ [^0\ ] ]]
  fi
}

used thus:

1: get_bad_things must succeed, but it should produce no output; but we want to see output that it does produce

get_bad_things | grep '^'
pipeinfo 0 1 || return

2: all pipeline must succeed

thing | something -q | thingy
pipeinfo || return

Answer 5

PIPESTATUS[@] must be copied to an array immediately after the pipe command returns. Any reads of PIPESTATUS[@] will erase the contents. Copy it to another array if you plan on checking the status of all pipe commands. "$?" is the same value as the last element of "${PIPESTATUS[@]}", and reading it seems to destroy "${PIPESTATUS[@]}", but I haven't absolutely verified this.

declare -a PSA  
cmd1 | cmd2 | cmd3  
PSA=( "${PIPESTATUS[@]}" )

This will not work if the pipe is in a sub-shell. For a solution to that problem,
see bash pipestatus in backticked command?

Answer 6

It may sometimes be simpler and clearer to use an external command, rather than digging into the details of bash. pipeline, from the minimal process scripting language execline, exits with the return code of the second command*, just like a sh pipeline does, but unlike sh, it allows reversing the direction of the pipe, so that we can capture the return code of the producer process (the below is all on the sh command line, but with execline installed):

$ # using the full execline grammar with the execlineb parser:
$ execlineb -c 'pipeline { echo "hello world" } tee out.txt'
hello world
$ cat out.txt
hello world

$ # for these simple examples, one can forego the parser and just use "" as a separator
$ # traditional order
$ pipeline echo "hello world" "" tee out.txt 
hello world

$ # "write" order (second command writes rather than reads)
$ pipeline -w tee out.txt "" echo "hello world"
hello world

$ # pipeline execs into the second command, so that's the RC we get
$ pipeline -w tee out.txt "" false; echo $?
1

$ pipeline -w tee out.txt "" true; echo $?
0

$ # output and exit status
$ pipeline -w tee out.txt "" sh -c "echo 'hello world'; exit 42"; echo "RC: $?"
hello world
RC: 42
$ cat out.txt
hello world

Using pipeline has the same differences to native bash pipelines as the bash process substitution used in answer #43972501.

* Actually pipeline doesn't exit at all unless there is an error. It executes into the second command, so it's the second command that does the returning.