Getting MongoDB error on Write - quota exceeded

泄露秘密 提交于 2019-12-30 11:02:14

问题


I have a JAVA app which writes to a replicaset. Am using 3.0.7 version of MongoDB server. The mongo-driver for Java is 3.0.4. It was working just fine but now am getting the following error on ALL writes:

com.mongodb.MongoWriteException: quota exceeded
at    com.mongodb.MongoCollectionImpl.executeSingleWriteRequest(MongoCollectionImpl.java:487)
at com.mongodb.MongoCollectionImpl.update(MongoCollectionImpl.java:474)
at com.mongodb.MongoCollectionImpl.updateOne(MongoCollectionImpl.java:325)

Have looked at the MongoDB config documentation and I have not set any quota limits in mongod.conf. Am not using smallFiles either.

But I think am running into fileSize limits. The file sizes for my DB are as follows:

total 6240548
-rw------- 1 mongod mongod   67108864 Dec  8 16:55 2015.0
-rw------- 1 mongod mongod  134217728 Dec  8 12:15 2015.1
-rw------- 1 mongod mongod  268435456 Dec  8 12:15 2015.2
-rw------- 1 mongod mongod  536870912 Dec  8 12:15 2015.3
-rw------- 1 mongod mongod 1073741824 Dec  8 12:15 2015.4
-rw------- 1 mongod mongod 2146435072 Dec  8 16:06 2015.5
-rw------- 1 mongod mongod 2146435072 Dec  8 16:06 2015.6
-rw------- 1 mongod mongod   16777216 Dec  8 16:55 2015.ns
drwxr-xr-x 2 mongod mongod       4096 Dec  7 09:14 _tmp

The /etc/mongod.conf is as follows:

    storage:
       dbPath: /data/mongoDB
       indexBuildRetry: true
       repairPath: /data/mongoDB/repair
       journal:
          enabled: true
       directoryPerDB: true
       syncPeriodSecs: 60
       engine: mmapv1
       mmapv1:
          preallocDataFiles: false
          nsSize: 16
          quota:
             enforced: false
             maxFilesPerDB: 8
          smallFiles: false
          journal:
             debugFlags: 0
             commitIntervalMs: 100

What can be going wrong?

PS: /etc/mongod.conf is being used.

mongod   10864     1  0 Nov16 ?        03:10:34 /usr/bin/mongod -f /etc/mongod.conf

Update- 1 .Tried the same update by changing collection name to a new collection. THat worked! but doesnt explain the issue yet. 2. Changed java driver to 3.0.3, that didnt help.

Update- 2:(12/9) Adding Collection Stats, since it is something to do with the collection itself and the java driver. Let me know if something seems awry pls.

    {
        "ns" : "2015.events",
        "count" : 827054,
        "size" : 3814018,
        "avgObjSize" : 4722,
        "numExtents" : 22,
        "extents" : [
            {
                "len" : 8192,
                "loc: " : {
                    "file" : 0,
                    "offset" : 20480
                }
            },
            {
                "len" : 32768,
                "loc: " : {
                    "file" : 0,
                    "offset" : 2134016
                }
            },
            {
                "len" : 131072,
                "loc: " : {
                    "file" : 0,
                    "offset" : 2166784
                }
            },
            {
                "len" : 524288,
                "loc: " : {
                    "file" : 0,
                    "offset" : 2297856
                }
            },
            {
                "len" : 2097152,
                "loc: " : {
                    "file" : 0,
                    "offset" : 2822144
                }
            },
            {
                "len" : 8388608,
                "loc: " : {
                    "file" : 0,
                    "offset" : 4919296
                }
            },
            {
                "len" : 11325440,
                "loc: " : {
                    "file" : 0,
                    "offset" : 14356480
                }
            },
            {
                "len" : 15290368,
                "loc: " : {
                    "file" : 0,
                    "offset" : 28827648
                }
            },
            {
                "len" : 20643840,
                "loc: " : {
                    "file" : 0,
                    "offset" : 44118016
                }
            },
            {
                "len" : 27869184,
                "loc: " : {
                    "file" : 1,
                    "offset" : 8192
                }
            },
            {
                "len" : 37625856,
                "loc: " : {
                    "file" : 1,
                    "offset" : 36265984
                }
            },
            {
                "len" : 50798592,
                "loc: " : {
                    "file" : 1,
                    "offset" : 78086144
                }
            },
            {
                "len" : 68579328,
                "loc: " : {
                    "file" : 2,
                    "offset" : 8396800
                }
            },
            {
                "len" : 92585984,
                "loc: " : {
                    "file" : 2,
                    "offset" : 93753344
                }
            },
            {
                "len" : 124993536,
                "loc: " : {
                    "file" : 3,
                    "offset" : 8192
                }
            },
            {
                "len" : 168742912,
                "loc: " : {
                    "file" : 3,
                    "offset" : 125001728
                }
            },
            {
                "len" : 227803136,
                "loc: " : {
                    "file" : 4,
                    "offset" : 8192
                }
            },
            {
                "len" : 307535872,
                "loc: " : {
                    "file" : 4,
                    "offset" : 239136768
                }
            },
            {
                "len" : 415174656,
                "loc: " : {
                    "file" : 4,
                    "offset" : 595939328
                }
            },
            {
                "len" : 560488448,
                "loc: " : {
                    "file" : 5,
                    "offset" : 8192
                }
            },
            {
                "len" : 756662272,
                "loc: " : {
                    "file" : 5,
                    "offset" : 607756288
                }
            },
            {
                "len" : 1021497344,
                "loc: " : {
                    "file" : 6,
                    "offset" : 8192
                }
            }
        ],
        "storageSize" : 3826952,
        "lastExtentSize" : 997556,
        "paddingFactor" : 1,
        "paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",
        "userFlags" : 1,
        "capped" : false,
        "nindexes" : 8,
        "indexDetails" : {

        },
        "totalIndexSize" : 348270,
        "indexSizes" : {
            "_id_" : 47123,
            "remoteRequest.uri_1" : 76969,
            "transactionId_1" : 59611,
            "startTime_1" : 39027,
            "endTime_1" : 35235,
            "remoteRequest.queryParams.q_1" : 20328,
            "remoteRequest.queryParams.fq_1" : 38285,
            "elapsedTimeInNanos_1" : 31689
        },
        "ok" : 1
    }

回答1:


Your mongod.conf has a quota enabled for each database. Based on that mongod.conf file, you will be unable to create more than 8 database files, which limits you to a max of about 6.4 GB of storage. You mention that you are able to get around this issue by using a new collection, so I am interested in what your data directory looks like now. I would not expect you to be able to bypass this hard limit, however due to internal data structures, it may be possible to "bypass" it for a short time.

You can verify how much actual data is being stored by running the dbStats command

use 2015
db.stats(1024*1024)

This output will tell you how much data you actually have in the db, vs the amount of allocated storage. These numbers will not match, this is expected as documents include empty space for padding.

My next question would be, is there are reason you are artificially limiting the amount of storage space your mongod can allocate? Perhaps a capped collection would better suit your needs? If you could expand on your use, I can perhaps give you a better answer.



来源:https://stackoverflow.com/questions/34166991/getting-mongodb-error-on-write-quota-exceeded

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!