Webshot to Google Drive without storing intermediate file, using buffers or streams?

问题

tl;dr

I am currently attempting to get a screenshot using webshot and upload to Google Drive without saving the file to the filesystem as an intermediate step in the process.

Any code--regardless of the approach--that will allow me to do this is most welcome!!

What I've tried

I was able to get the system to run locally by saving the file from webshot and then uploading that file to Google Drive, but this is not possible on the server environment I use (Elastic Beanstalk), and I would like to avoid the messiness regardless.

Webshot allows streaming the screenshot. Here's their sample code:

var webshot = require('webshot');
var fs      = require('fs');

var renderStream = webshot('google.com');
var file = fs.createWriteStream('google.png', {encoding: 'binary'});

renderStream.on('data', function(data) {
  file.write(data.toString('binary'), 'binary');
});

However, this seems to accomplish the same purpose as just writing to file.

Google Drive, in their google-api-nodejs-client (which I use) allows setting the body to a readable stream. Here is their sample code:

var fs = require('fs');
var drive = google.drive({ version: 'v3', auth: oauth2Client });

drive.files.create({
  resource: {
    name: 'testimage.png',
    mimeType: 'image/png'
  },
  media: {
    mimeType: 'image/png',
    body: fs.createReadStream('awesome.png') // read streams are awesome!
  }
}, callback);

It seems to me that since both fs.createWriteStream and fs.createReadStream allow writing to a Buffer, I should be able to set one up to both write to and read from. However, I have tried every possible combination of doing this, searched the web and StackOverflow for examples, and have attempted suggested solutions that came even somewhat close to my specific application. Webshot discusses uploading to S3 as a stream [Note: I tried posting a third link here, but don't have enough reputation points. To find it, you will have to look at issue #90 in the node-webshot repo], though I couldn't get that to work with Google Drive. I have tried everything I can think of, and posting to StackOverflow is my last resort. Here is the current (failing) code I have, though I could be way off-base:

let buffers = [];
let renderStream = webshot('google.com');
renderStream.on('data', chunk=> buffers.push(chunk));
renderStream.on('end', ()=> {
  drive.files.create({
    auth: oauth2Client,
    resource: {
      name: 'testimage.png',
      mimeType: 'image/png',
      parents: [ folderID ],
    },
    media: {
      mimeType: 'image/png',
      body: fs.createReadStream(Buffer.concat(buffers))
    }
  }, (error, response, body)=> {
    if (error || !response || !response.name) {
      console.log(error, response);
    }
  });
});

[NOTE: This is a simplification of the code. The drive.files.create() portion is actually contained within its own function which 100% works when passed an actual PNG file (that is used in media.body in place of the fs.createReadStream(Buffer.concat(buffers))), even one downloaded from webshot]

I'm fairly new to the stream/buffer world, as may be obvious. However, I'm at my wit's end on this problem and will take any help I can get!!

Thank you in advance for any suggestions.

回答1:

Ironically, I was inches from the finish line. In fact, as I was closing out my tabs after posting this question, I noticed that the test file had shown up in Google Drive! My last implementation had worked: I just saw an error and assumed it hadn't. (The error was due to the fact that I kept an fs.unlink procedure in there to delete the intermediate file...that I've now completely avoided!)

Here is the end result (note that I've combined two different modules here so that you can see everything at a glance):

'use strict';

const gdrive     = require('googleapis').drive('v3');
const googleAuth = require('google-auth-library');
const webshot    = require('webshot');

const creds = require('path/to/client_secret.json').installed;
const auth  = new googleAuth();

let drive = {};
let submission = {};

drive.oauth2Client = new auth.OAuth2(creds.client_id, creds.client_secret, creds.redirect_uris[0]);
drive.oauth2Client.credentials = require('path/to/drive.json');

drive.uploadStream = (stream, filename, folderID, callback)=> {
  gdrive.files.create({
    auth: drive.oauth2Client,
    resource: {
      name: filename,
      mimeType: 'image/png',
      parents: [ folderID ],
    },
    media: {
      mimeType: 'image/png',
      body: stream,
    }
  }, callback);
};

submission.submit = (obj, callback)=> {
  let buffers = [];
  let renderStream = webshot(obj.uri);
  renderStream.on('data', chunk=> buffers.push(chunk));
  renderStream.on('end', ()=> {
    drive.uploadStream(Buffer.concat(buffers), `${obj.filename}.png`, obj.folderID, callback);
  });
};

/*
  Example:

  submission.submit({ uri: 'https://duckduckgo.com', filename: 'ddg', folderID: 'G0bBl3deg00K' }, (err, response, body)=> {
    console.log(err || response);
  });
*/

来源：https://stackoverflow.com/questions/38901838/webshot-to-google-drive-without-storing-intermediate-file-using-buffers-or-stre

标签

node.js

stream

google-drive-api

buffer