node.js-stream

I basically want to do something like this: $ my-node-cli <some-param> | less Note that less is just an example. I need it to work with any other *nix command. More about the use case: I wrote a node CLI package that searches some online resource and outputs results to the shell. Since the result set can be huge, client wants to do additional operations on it, e.g grep , head , tail , tee , ... anything really. I searched far and wide and I only managed to find the way to pipe into node program, not out of . My current idea is to capture the right side of pipe when my program is called, then,

(new information below) I am trying to set up a lambda function that reacts to uploaded tgz files by uncompressing them and writing the results back to S3. The unzip and untar work fine, but uploading to S3 fails: /Users/russell/lambda/gzip/node_modules/aws-sdk/lib/s3/managed_upload.js:350 var buf = self.body.read(self.partSize - self.partBuffer.length) || ^ TypeError: undefined is not a function at ManagedUpload.fillStream (/Users/russell/lambda/gzip/node_modules/aws-sdk/lib/s3/managed_upload.js:350:25) at Entry.<anonymous> (/Users/russell/lambda/gzip/node_modules/aws-sdk/lib/s3/managed

As the documentation says, they both deal with transforming non-stream plugins to stream. What I try to understand is, if I can use the .pipe() method on something, doesn't it mean it's a stream? If so, what do I convert to what here? vinyl-source-stream example: (from: https://www.npmjs.com/package/vinyl-buffer ) var browserify = require('browserify') var source = require('vinyl-source-stream') var buffer = require('vinyl-buffer') var uglify = require('gulp-uglify') var size = require('gulp-size') var gulp = require('gulp') gulp.task('build', function() { var bundler = browserify('./index.js'

I want to pipe data from an amazon kinesis stream to a an s3 log or a bunyan log. The sample works with a file write stream or stdout. How would I implmeny my own writable stream? //this works var file = fs.createWriteStream('my.log') kinesisSource.pipe(file) this doesn't work saying it has no method 'on' var stream = {}; //process.stdout works however stream.writable = true; stream.write =function(data){ console.log(data); }; kinesisSource.pipe(stream); what methods do I have to implement for my own custom writable stream, the docs seem to indicate I need to implement 'write' and not 'on'

I'd like to combine two Node.js streams into one by piping them, if possible. I'm using Transform streams. In other words, I'd like my library to return myStream for people to use. For example they could write: process.stdin.pipe(myStream).pipe(process.stdout); And internally I'm using a third-party vendorStream that does some work, plugged into my own logic contained in myInternalStream . So what's above would translate to: process.stdin.pipe(vendorStream).pipe(myInternalStream).pipe(process.stdout); Can I do something like that? I've tried var myStream = vendorStream.pipe(myInternalStream)

The goal is to: Create a file read stream. Pipe it to gzip ( zlib.createGzip() ) Then pipe the read stream of zlib output to: 1) HTTP response object 2) and writable file stream to save the gzipped output. Now I can do down to 3.1: var gzip = zlib.createGzip(), sourceFileStream = fs.createReadStream(sourceFilePath), targetFileStream = fs.createWriteStream(targetFilePath); response.setHeader('Content-Encoding', 'gzip'); sourceFileStream.pipe(gzip).pipe(response); ... which works fine, but I need to also save the gzipped data to a file so that I don't need to regzip every time and be able to

I'm trying to understand node streams and their life-cycle. So, I want to split the content of a stream for n-parts. The code below is just to explain my intentions and to show that I already try something by myself. I omitted some details I have a stream which just generates some data(just a sequence of numbers): class Stream extends Readable { constructor() { super({objectMode: true, highWaterMark: 1}) this.counter = 0 } _read(size) { if(this.counter === 30) { this.push(null) } else { this.push(this.counter) } this.counter += 1 } } const stream = new Stream() stream.pause(); a function which