How to set Buffer offset range in MongoDB, Its not allowing to upload more than 16MB file in BSON Object?

问题

My Flow and Code:

Uploading image from Form Data and storing image's BSON string to Database, below are the steps and code:

1. Uploading file from multer upload
2. Reading file from filesystem - fs
3. Converting image content string to base64 string
4. Converting base64 string to BSON String

Configurations:

node -v: v12.13.1
npm -v: 6.12.1
fs-extra: ^8.1.0
multer: ^1.4.2

Code:

var upload = multer({ 
    storage: multer.diskStorage({
        destination: function (req, file, cb) {
            cb(null, 'uploads')
        },
        filename: function (req, file, cb) {
            cb(null, file.fieldname + '-' + Date.now())
        }
    })
});
upload.single('picture'), (req, res) => {

    let imageString = fs.readFileSync(req.file.path);
    let encodeImage = imageString.toString('base64');
    let bufferImage = Buffer.from(encodeImage, 'base64');
    var finalObj = {
        contentType: req.file.mimetype,
        image: bufferImage
    };
    db.collection('filesUpload').insertOne(finalObj, (err, result) => {
        if (err) {
            console.log(err);
        }else{
            console.log('success');
        }
    });

});

What is working? I can able to upload below 16MB images successful and can read and retrieve it properly from Database.

What is not Working?

I can not able to upload above 16MB images.

Error:

RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 && <= 17825792. Received 18646861
at Buffer.write (buffer.js:1019:5)
at serializeObjectId (D:\api\node_modules\mongodb\node_modules\bson\lib\bson\parser\serializer.js:274:14)    at serializeInto (D:\api\node_modules\mongodb\node_modules\bson\lib\bson\parser\serializer.js:935:17)    
at serializeObject (D:\api\node_modules\mongodb\node_modules\bson\lib\bson\parser\serializer.js:347:18)  
at serializeInto (D:\api\node_modules\mongodb\node_modules\bson\lib\bson\parser\serializer.js:727:17)    
at serializeObject (D:\api\node_modules\mongodb\node_modules\bson\lib\bson\parser\serializer.js:347:18)  
at serializeInto (D:\api\node_modules\mongodb\node_modules\bson\lib\bson\parser\serializer.js:941:17)    
at BSON.serialize (D:\api\node_modules\mongodb\node_modules\bson\lib\bson\bson.js:64:28)
at Msg.serializeBson (D:\api\node_modules\mongodb\lib\core\connection\msg.js:126:22)
at Msg.makeDocumentSegment (D:\api\node_modules\mongodb\lib\core\connection\msg.js:118:33)
at Msg.toBin (D:\api\node_modules\mongodb\lib\core\connection\msg.js:104:25)
at serializeCommand (D:\api\node_modules\mongodb\lib\core\connection\pool.js:779:41)
at Pool.write (D:\api\node_modules\mongodb\lib\core\connection\pool.js:927:3)
at _command (D:\api\node_modules\mongodb\lib\core\wireprotocol\command.js:128:10)
at command (D:\api\node_modules\mongodb\lib\core\wireprotocol\command.js:28:5)
at writeCommand (D:\api\node_modules\mongodb\lib\core\wireprotocol\write_command.js:47:3) {
    code: 'ERR_OUT_OF_RANGE'
}

I am not getting how to solve this, i think its MongoDB error, thanks for your help.

回答1:

Finally i found answer for my question:

Here i have used Binary JSON called BSON Document, So there are limits to store in MongoDB as per MongoDB Docs:

• The maximum BSON document size is 16 megabytes
• MongoDB supports no more than 100 levels of nesting for BSON documents

What is the other replacement method?

GridFS:

• Definition:

As per MongoDB documentation MongoDB GridFS Docs:

GridFS is a specification for storing and retrieving files that exceed the BSON-document size limit of 16 MB. GridFS does not support multi-document transactions. Instead of storing a file in a single document, GridFS divides the file into parts, or chunks, and stores each chunk as a separate document.

• API Reference:

If you need to implement this is NodeJS then MongoDB has provided Docs, Here is the MongoDB API Reference Docs

• Stack Overflow Reference:

If you need any help in implementation then refer stack overflow post, Here is the Stack Overflow Reference

来源：https://stackoverflow.com/questions/62129964/how-to-set-buffer-offset-range-in-mongodb-its-not-allowing-to-upload-more-than