Uploading large file to NodeJS webserver

Question

I am serving up my web application using NodeJS server (ExpressJS) currently. One of the new requirements is for the users to be able to upload large videos (potentially in gigs

Answer 1

    var express =   require("express");

    var app =   express();
    var async = require('async');
    var fs = require('fs');
    var client = redis.createClient();
    var multiparty = require('multiparty');
    var util = require('util');

    app.post('/',function(req,res){

        var form = new multiparty.Form();

        form.parse(req, function(err, fields, files) {

             client.get(fields.otp, function(err, reply) {
    var oldpath = files.file[0].path;
                          var newpath = path + files.file[0].originalFilename;
                          fs.rename(oldpath, newpath, function (err) {
                            if (err) throw err;
                            res.write('File uploaded and moved!');
                            res.end();
                          });
    }
    }



app.listen(2001,function(){
    console.log("Server is running on port 2001");
});

Answer 2

Multer should work fine. It'll stream the data as it comes in over HTTP to a file on disk. It won't lock up your server. Then in your endpoint you have access to that file location and can do whatever you need with it.

Answer 3

Having done this myself (~20GB files, without multer) I can recommend the following (Probably most of which you have considered, but for completeness):

1. Choose an appropriate upload control (or write your own, but basically something that chunks up the data is best. I used plupload I think)

2. On the server make an API to handle the received chunk data, and write it out to a temp location during upload (You may want to hash and check individual pieces as well).

3. Once upload complete, check file (I used a hash generated by the client, and checked it against the uploaded data)