Node.js spawning a child process interactively with separate stdout and stderr streams

Question

Consider the following C program (test.c):

#include <stdio.h>

int main() {
  printf("string out 1\n");
  fprintf(stderr, "string err 1\n");
  getchar();
  printf("string out 2\n");
  fprintf(stderr, "string err 2\n");
  fclose(stdout);
}

Which should print a line to stdout, a line to stderr, then wait for user input, then another line to stdout and another line to stderr. Very basic! When compiled and run on the command line the output of the program when complete (user input is received for getchar()):

$ ./test 
string out 1
string err 1

string out 2
string err 2

When trying to spawn this program as a child process using nodejs with the following code:

var TEST_EXEC = './test';

var spawn = require('child_process').spawn;
var test = spawn(TEST_EXEC);

test.stdout.on('data', function (data) {
  console.log('stdout: ' + data);
});

test.stderr.on('data', function (data) {
  console.log('stderr: ' + data);
});

// Simulate entering data for getchar() after 1 second
setTimeout(function() {
  test.stdin.write('\n');
}, 1000);

The output appears like this:

$ nodejs test.js 
stderr: string err 1

stdout: string out 1
string out 2

stderr: string err 2

Very different from the output as seen when running ./test in the terminal. This is because the ./test program isn't running in an interactive shell when spawned by nodejs. The test.c stdout stream is buffered and when run in a terminal as soon as a \n is reached the buffer is flushed but when spawned in this way with node the buffer isn't flushed. This could be resolved by either flushing stdout after every print, or changing the stdout stream to be unbuffered so it flushes everything immediately. Assuming that test.c source isn't available or modifiable, neither of the two flushing options mentioned can be implemented.

I then started looking at emulating an interactive shell, there's pty.js (pseudo terminal) which does a good job, for example:

var spawn = require('pty.js').spawn;
var test = spawn(TEST_EXEC);

test.on('data', function (data) {
  console.log('data: ' + data);
});

// Simulate entering data for getchar() after 1 second
setTimeout(function() {
  test.write('\n');
}, 1000);

Which outputs:

$ nodejs test.js
data: string out 1
string err 1

data: 

data: string out 2
string err 2

However both stdout and stderr are merged together (as you would see when running the program in a terminal) and I can't think of a way to separate the data from the streams.

So the question..

Is there any way using nodejs to achieve the output as seen when running ./test without modifying the test.c code? Either by terminal emulation or process spawning or any other method?

Cheers!

Answer 1

I tried the answer by user568109 but this does not work, which makes sense since the pipe only copies the data between streams. Hence, it only gets to process.stdout when the buffer is flushed... The following appears to work:

var TEST_EXEC = './test';

var spawn = require('child_process').spawn;
var test = spawn(TEST_EXEC, [], { stdio: 'inherit' });

//the following is unfortunately not working 
//test.stdout.on('data', function (data) {
//  console.log('stdout: ' + data);
//});

Note that this effectively shares stdio's with the node process. Not sure if you can live with that.

Answer 2

I was just revisiting this since there is now a 'shell' option available for the spawn command in node since version 5.7.0. Unfortunately there doesn't seem to be an option to spawn an interactive shell (I also tried with shell: '/bin/sh -i' but no joy). However I just found this which suggests using 'stdbuf' allowing you to change the buffering options of the program that you want to run. Setting them to 0 on everything produces unbuffered output for all streams and they're still kept separate.

Here's the updated javascript:

var TEST_EXEC = './test';

var spawn = require('child_process').spawn;
var test = spawn('stdbuf', ['-i0', '-o0', '-e0', TEST_EXEC]);

test.stdout.on('data', function (data) {
  console.log('stdout: ' + data);
});

test.stderr.on('data', function (data) {
  console.log('stderr: ' + data);
});

// Simulate entering data for getchar() after 1 second
setTimeout(function() {
  test.stdin.write('\n');
}, 1000);

Looks like this isn't pre-installed on OSX and of course not available for Windows, may be similar alternatives though.

Answer 3

You can do this :

var TEST_EXEC = 'test';
var spawn = require('child_process').spawn;
var test = spawn(TEST_EXEC);

test.stdin.pipe(process.stdin);
test.stdout.pipe(process.stdout);
test.stderr.pipe(process.stderr);

When you use events on stdout and stderr to print the output on console.log, you will get jumbled output because of asynchronous execution of the functions. The output will be ordered for a stream independently, but output can still get interleaved among stdin,stdout and stderr.