Creating incrementing numbers with mongoDB

Question

We have an order system where every order has an id. For accounting purposes we need a way to generate invoices with incremening numbers. What is the best way to do this without

Answer 1

I did not find any working solution, so I implemented the "optimistic loop" in node.js to get Auto-Incrementing Interger ID fields. Uses the async module to realize the while loop.

// Insert the document to the targetCollection. Use auto-incremented integer IDs instead of UIDs. 
function insertDocument(targetCollection, document, callback) {
    var keepRunning = true;
    var seq = 1;
    // $type 16/18: Integer Values
    var isNumericQuery = {$or : [{"_id" : { $type : 16 }}, {"_id" : { $type : 18 }}]};
    async.whilst(testFunction, mainFunction, afterFinishFunction);
    // Called before each execution of mainFunction(). Works like the stop criteria of a while function.
    function testFunction() { 
        return keepRunning; 
    }
    // Called each time the testFunction() passes. It is passed a function (next) which must be called after it has completed.
    function mainFunction(next) {
        findCursor(targetCollection, findCursorCallback, isNumericQuery, { _id: 1 });
        function findCursorCallback(cursor) {
            cursor.sort( { _id: -1 } ).limit(1);
            cursor.each(cursorEachCallback);
        }
        function cursorEachCallback(err, doc) {
            if (err) console.error("ERROR: " + err);
            if (doc != null) {
                seq = doc._id + 1;
                document._id = seq;
                targetCollection.insert(document, insertCallback);
            }
            if (seq === 1) {
                document._id = 1;
                targetCollection.insert(document, insertCallback);
            }
        }
        function insertCallback(err, result) {
            if (err) {
                console.dir(err);
            }
            else {
                keepRunning = false;
            }
            next();
        }
    }
    // Called once after the testFunction() fails and the loop has ended. 
    function afterFinishFunction(err) {
        callback(err, null);
    }
}

// Call find() with optional query and projection criteria and return the cursor object.
function findCursor(collection, callback, optQueryObject, optProjectionObject) {
    if (optProjectionObject === undefined) {
        optProjectionObject = {};
    }
    var cursor = collection.find(optQueryObject, optProjectionObject);
    callback(cursor);
}

Call with

insertDocument(db.collection(collectionName), documentToSave, function() {if(err) console.error(err);});

Answer 2

http://www.mongodb.org/display/DOCS/How+to+Make+an+Auto+Incrementing+Field

The first approach is keeping counters in a side document:

One can keep a counter of the current _id in a side document, in a collection dedicated to counters. Then use FindAndModify to atomically obtain an id and increment the counter.

The other approach is to loop optimistically and handle dup key error code of 11000 by continuing and incrementing the id for the edge case of collisions. That works well unless there's high concurrency writes to a specific collection.

One can do it with an optimistic concurrency "insert if not present" loop.

But be aware of the warning on that page:

Generally in MongoDB, one does not use an auto-increment pattern for _id's (or other fields), as this does not scale up well on large database clusters. Instead one typically uses Object IDs.

Other things to consider:

• Timestamp - unique long but not incrementing (base on epoch)
• Hybrid Approach - apps don't necessarily have to pick one storage option.
• Come up with your id mechanism based on things like customer, date/time parts etc... that you generate and handle collisions for. Depending on the scheme, collisions can be much less likely. Not necessarily incrementing but is unique and has a well defined readable pattern.