How can I split and re-join STDOUT from multiple processes?

Question

I am working on a pipeline that has a few branch points that subsequently merge-- they look something like this:

         command2
        /        \\
command1           


        

Answer 1

You can play around with file descriptors like this;

((date | tee >( wc >&3) | wc) 3>&1) | wc

or

((command1 | tee >( command2 >&3) | command3) 3>&1) | command4

To explain, that is tee >( wc >&3) will output the original data on stdout, and the inner wc will output the result on FD 3. The outer 3>&1) will then merge FD3 output back into STDOUT so output from both wc is sent to the tailing command.

HOWEVER, there is nothing in this pipeline (or the one in your own solution) which will guanrantee that the output will not be mangled. That is incomplete lines from command2 will not be mixed up with lines of command3 -- if that is a concern, you will need to do one of two things;

1. Write your own tee program which internally uses popen and read each line back before sending complete lines to stdout for command4 to read
2. Write the output from command2 and command3 to a file and use cat to merge the data as input to command4