Is there a MongoDB maximum bson size work around?

Question

The document I am working on is extremely large. It collects user input from an extremely long survey (like survey monkey) and stores the answers in a mongodb database.

Answer 1

You should be using gridfs. It allows you to store documents in chunks. Here's the link: http://docs.mongodb.org/manual/reference/gridfs/

Answer 2

One thing you can do is to build your own mongoDB :-). Mongodb is an open source and the limitation about the size of a document is rather arbitrary to enforce a better schema design. You can just modify this line and build it for yourself. Be careful with this.

The most straight forward idea is to have each small question in a different document with a field which reference to its parent.

Another idea is to limit number of documents in the parent. Lets say you limit is N elements then the parent looks like this:

{
  _id : ObjectId(),
  id : { type: Number, required: true },
  created: { type: Date, default: Date.now },  // you can store it only for the first element
  last_modified: { type: Date, default: Date.now }, // the same here
  data : [{
    id: 65,
    question: {
        test: "some questions",
        answers: [2,5,6]
    }
  }, ... up to N of such things {}
  ]
}

This way modifying number N you can make sure that you will be in 16 MB of BSON. And in order to read the whole survey you can select

db.coll.find({id: the Id you need}) and then combine the whole survey on the application level. Also do not forget to ensureIndex on id.

Try different things, do a benchmark on your data and see what works for you.