Get uploaded file as stream using Express

问题

Right now, I'm trying to make an API, and one of its functions is to take a file and upload it to another website, sort of like uploading a file through the API as some sort of "proxy".

The website I'm sending the file to requires additional steps to work (namely sending the destination in another request) and I wanted to have my API do that all in one request instead, where that request already had all the details needed.

I've gotten the request for the destination done, but I'm having a hard time finding out how to "pipe" the upload of a user into the request that I send to the website. request-promise takes a file stream as an input when uploading files, so I wanted to hopefully pipe the upload coming from the user to the request. That way I don't have to use any disk space to temporarily save the file, especially since I'm very limited in disk space and since the server I'm sending it to virtually has no file limit.

As of now, I'm using express-fileupload to temporarily save the file, but this will eventually blow up my extremely small (194 MB) allocated space. I'm using express v4.16.4 and Node v12.0.0. For the requests, I use request-promise v4.2.4.

Here's how I declared the Express app and the express-fileupload object.

var router = express.Router({ caseSensitive: true });
app.use(router);

...

router.use(require("express-fileupload")({
    useTempFiles : true,
    tempFileDir : ".data/temp"
}));

And here's what I'm using to toss the file into the website:

if (req.body.id === undefined || req.files.file === undefined)
    return new APIError(400);
if (!Object.keys(JSON.parse(fs.read(".data/master_list_of_possible_values_for_that_variable.json"))).includes(req.body.id))
    return new APIError("400-ID");

var session_flag = await(session.executeRequest({
    method: "POST",
    uri: Constants.URL.upload.session,
    form: {
        id: req.body.id
    }
})
    .then(r => {return r;})
    .catch(e => { return e; }));

if (session_flag instanceof Error)
    throw session_flag;

var upload = await(session.executeRequest({
    method: "POST",
    uri: Constants.URL.upload.fileEndpoint,
    formData: {
        file: {
            value: fs.createReadStream(req.files.file.tempFilePath),
            options: {
                filename: req.files.file.name,
                contentType: req.files.file.mimetype
            }
        }
    }
})
    .then(r => {return r;})
    .catch(e => { return e; }));

if (upload instanceof Error)
    throw upload;

return { success: true };

回答1:

What I suggest you do, is the following:

Change your endpoint so you send id in the URL, that way you don't need to read the whole body to issue the first request
Pipe the request directly to the third party endpoint

const request = require('request'); // streaming works better with request
const rp = require('request-promise');
const { once } = require('events');

// Make sure you don't have any body parser enable for this request
router.post('/:id', async(req, res) => {
   try {

        // issue first request
        await rp({
            method: "POST",
            uri: Constants.URL.upload.session,
            form: {
                id: req.body.id
            }
        })
        // pipe request
        const upload = req.pipe(request({
            method: "POST",
            uri: Constants.URL.upload.fileEndpoint,
            headers: req.headers // Send Same Content-Type, Length and so on
        }))

        // will throw on 'error'
        await once(upload, 'response')

        // check status code if you want
        const success = upload.response.statusCode === 200;

        return res.json({ success })

   } catch(e) {
      res.status(500)
        .send('error')
   }
});

Doing this, you don't need to store it in a temporary file since you're piping it directly to the third party endpoint.

来源：https://stackoverflow.com/questions/59164411/get-uploaded-file-as-stream-using-express

标签

node.js

express

http

file-upload

upload