Google Cloud Storage - Error during upload: gcs-resumable-upload.json renaming operation not permitted

孤街醉人 提交于 2019-12-13 19:01:42

问题


I'm simply trying to follow this tutorial on how to upload files to gcs with Node and Express. But the following error keep causing my app to crash. Usually, I am able to upload one file without a problem in the first run. But I will get this error after running a few request, even with different file. When I try to upload, say 5, files at a time, this error cause my app to crash even in the first run. I see the process is trying to rename a file in the .config folder. Is it a normal behavior? If so, is there a work-around?

Window: v10.0.10586 Node: v4.3.1 Express: v4.13.1

Error: EPERM: operation not permitted, rename 'C:\Users\James Wang.config\configstore\gcs-resumable-upload.json.2873606827' -> 'C:\Users\James Wang.config\configstore\gcs-resumable-upload.json' at Error (native) at Object.fs.renameSync (fs.js:681:18) at Function.writeFileSync as sync at Object.create.all.set (C:\Users\James Wang\gi-cms-backend\node_modules\configstore\index.js:62:21) at Object.Configstore.set (C:\Users\James Wang\gi-cms-backend\node_modules\configstore\index.js:93:11) at Upload.set (C:\Users\James Wang\gi-cms-backend\node_modules\gcs-resumable-upload\index.js:264:20) at C:\Users\James Wang\gi-cms-backend\node_modules\gcs-resumable-upload\index.js:60:14 at C:\Users\James Wang\gi-cms-backend\node_modules\gcs-resumable-upload\index.js:103:5 at Request._callback (C:\Users\James Wang\gi-cms-backend\node_modules\gcs-resumable-upload\index.js:230:7) at Request.self.callback (C:\Users\James Wang\gi-cms-backend\node_modules\request\request.js:199:22) at emitTwo (events.js:87:13) at Request.emit (events.js:172:7) at Request. (C:\Users\James Wang\gi-cms-backend\node_modules\request\request.js:1036:10) at emitOne (events.js:82:20) at Request.emit (events.js:169:7) at IncomingMessage. (C:\Users\James Wang\gi-cms-backend\node_modules\request\request.js:963:12) [nodemon] app crashed - waiting for file changes before starting...

UPDATE: After setting {resumable: false} as suggested by @stephenplusplus in this post, I am no longer getting the "EPERM: operation not permitted" error.But, I start running into the { [ERROR:ETIMEDOUT] code: 'ETIMEDOUT', connection: false } error while trying to upload multiple files at a time with the largest file greater than 1.5mb. Other files get uploaded successfully.

For more information, I am able to upload files one by one when the files are no greater than ~2.5mb. If I try to upload 3 files at a time, I can only do so with files no greater than ~1.5mb.

Is the "Operation not permitted" issue as specified in the question a window specific thing, and does the timeout issue happen only after i set resumable = false?

I'm using express and multer with node.

This is the code I'm using now:

  // Express middleware that will handle an array of files. req.files is an array of files received from 
// filemulter.fields([{field: name, maxCount}]) function. This function should handle 
// the upload process of files asychronously
function sendFilesToGCS(req, res, next) {
    if(!req.files) { return next(); }

    function stream(file, key, folder) {
        var gcsName = Date.now() + file.originalname;
        var gcsFile = bucket.file(gcsName);
        var writeStream = gcsFile.createWriteStream({ resumable: false });
        console.log(key);
        console.log('Start uploading: ' + file.originalname);

        writeStream.on('error', function(err) {
            console.log(err);
            res.status(501).send(err);
        });
        writeStream.on('finish', function() {
            folder.incrementFinishCounter();
            req.files[key][0].cloudStorageObject = gcsName;
            req.files[key][0].cloudStoragePublicUrl = getPublicUrl(gcsName);
            console.log('Finish Uploading: ' + req.files[key][0].cloudStoragePublicUrl);
            folder.beginUploadNext();
        });
        writeStream.end(file.buffer);
    };

    var Folder = function(files) {
        var self = this;
        self.files = files;
        self.reqFilesKeys = Object.keys(files); // reqFilesKeys is an array of keys parsed from req.files
        self.nextInQuene = 0; // Keep track of the next file to be uploaded, must be less than reqFilesKeys.length
        self.finishCounter = 0; // Keep track of how many files have been uploaded, must be less than reqFilesKeys.length

        console.log(this.reqFilesKeys.length + ' files to upload');
    };

    // This function is used to initiate the upload process. 
    // It's also called in the on-finish listener of a file's write-stream,
    // which will start uploading the next file in quene
    Folder.prototype.beginUploadNext = function() {
        // If there's still file left to upload,
        if(this.finishCounter < this.reqFilesKeys.length) {
            // and if there's still file left in quene
            if(this.nextInQuene < this.reqFilesKeys.length) {
                // upload the file
                var fileToUpload = this.files[this.reqFilesKeys[this.nextInQuene]][0];
                stream(fileToUpload, this.reqFilesKeys[this.nextInQuene], this);
                // Increment the nextInQuene counter, and get the next one ready
                this.nextInQuene++;             
            }
        } else {
            console.log('Finish all upload!!!!!!!!!!!!!!!!!!!!!!');
            next();
        }
    };

    Folder.prototype.incrementFinishCounter = function() {
        this.finishCounter++;
        console.log('Finished' + this.finishCounter + ' files');
    };

    var folder = new Folder(req.files);

    // Begin upload with 3 streams
    /*for(var i=0; i<3; i++) {
        folder.beginUploadNext();
    }*/

    //Upload file one by one
    folder.beginUploadNext();
}

回答1:


I had the same issue with bower .. Run the following command: bower cache clean --allow-root

if this does not solve the problem, try after disabling anti virus.



来源:https://stackoverflow.com/questions/35853519/google-cloud-storage-error-during-upload-gcs-resumable-upload-json-renaming-o

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!