Why does 2>&1 need to come before a | (pipe) but after a “> myfile” (redirect to file)?

Question

When combining stderr with stdout, why does 2>&1 need to come before a | (pipe) but after a > myfile (redirect to file)?

To redirect stderr to stdout for file output:

  echo > myfile 2>&1

To redirect stderr to stdout for a pipe:

  echo 2>&1 | less

My assumption was that I could just do:

  echo | less 2>&1 

and it would work, but it doesn't. Why not?

Answer 1

A pipeline is a |-delimited list of commands. Any redirections you specify apply to the constituent commands (simple or compound), but not to the pipeline as a whole. Each pipe chains one command's stdout to the stdin of the next by implicitly applying a redirect to each subshell before any redirects associated with a command are evaluated.

cmd 2>&1 | less

First stdout of the first subshell is redirected to the pipe from which less is reading. Next, the 2>&1 redirect is applied to the first command. Redirecting stderr to stdout works because stdout is already pointing at the pipe.

cmd | less 2>&1

Here, the redirect applies to less. Less's stdout and stderr both presumably started out pointed at the terminal, so 2>&1 in this case has no effect.

If you want a redirect to apply to an entire pipeline, to group multiple commands as part of a pipeline, or to nest pipelines, then use a command group (or any other compound command):

{ { cmd1 >&3; cmd2; } 2>&1 | cmd3; } 3>&2

Might be a typical example. The end result is: cmd1 and cmd2's stderr -> cmd3; cmd2's stdout -> cmd3; and cmd1 and cmd3's stderr, and cmd3's stdout -> the terminal.

If you use the Bash-specific |& pipe, things get stranger, because each of the pipeline's stdout redirects still occur first, but the stderr redirect actually comes last. So for example:

f() { echo out; echo err >&2; }; f >/dev/null |& cat

Now, counterintuitively, all output is hidden. First stdout of f goes to the pipe, next stdout of f is redirected to /dev/null, and finally, stderr is redirected to stdout (/dev/null still).

I recommend never using |& in Bash -- it's used here for demonstration.

• Many of the obligatory redirection links

Answer 2

To add to ormaaj's answer:

The reason you need to specify redirection operators in the proper order is that they're evaluated from left to right. Consider these command lists:

# print "hello" on stdout and "world" on stderr
{ echo hello; echo world >&2; }

# Redirect stdout to the file "out"
# Then redirect stderr to the file "err"
{ echo hello; echo world >&2; } > out 2> err

# Redirect stdout to the file "out"
# Then redirect stderr to the (already redirected) stdout
# Result: all output is stored in "out"
{ echo hello; echo world >&2; } > out 2>&1

# Redirect stderr to the current stdout
# Then redirect stdout to the file "out"
# Result: "world" is displayed, and "hello" is stored in "out"
{ echo hello; echo world >&2; } 2>&1 > out

Answer 3

My answer is by understanding file descriptors. Each process has a bunch of file descriptors: entries to files that are opened. By default, number 0 is for stdin, number 1 is for stdout and number 2 is for stderr.

The i/o redirectors > and < by default connect to their most reasonable file descriptors, stout and stdin. If you re-route stdout to a file (as with foo > bar), on starting process 'foo', the file 'bar' is opened for writing and hooked on file descriptor number 1. If you want only stderr in 'bar', you'd use foo 2> bar which opens file bar and hooks it to the stderr.

Now the i/o redirector '2>&1'. I normally read that as 'put file descriptor 2 to the same as file descriptor 1. While reading the commandline from left to right, you can do the next: foo 1>bar 2>&1 1>/dev/tty. With this, file descriptor 1 is set to the file 'bar', file descriptor 2 is set to the same as 1 (hence 'bar') and after that, file descriptor 1 is set to /dev/tty. The runnning foo is sending its output to /dev/tty and it stderr to the file 'bar'.

Now the pipeline comes in: this does not alter the file descriptors, however, it will connect them between the processes: stdout of the left process ot stdin of the next. Stderr is passed on. Hence, if you like the pipeline to work on stderr only, you use foo 2| bar, which connects stderr of foo to stdin of bar. (I'm not sure what happens with the stdout of foo.)

With the above, if you use foo 2>&1 | bar, since stderr of foo is re-routed to stdout of foo, both stdout and stderr of foo arrive at the stdin of bar.