How could I write aggregation without exceeds maximum document size?

Question

I got exceeds maximum document size problem exception by the query as follows,

pipe = [
    {\"$match\": { \"birthday\":{\"$gte\":datetime.datet         


        

Answer 1

As @Frederick said requires mongo 2.6 at least, For further reference, here is the link from mongo documentation, which works similar to runCommand way but with db.collection.aggreagate, note that for document limit use "cursor" option, for sort limit use "allowDiskUse" option.

Answer 2

By default the result of aggregations are returned to you in a single BSON document, which is where the size restriction comes from. If you need to return more than that you can either:

• have the results be output to a collection. You do this by finishing your pipeline with

  {"$out": "some-collection-name"}

  You then query that collection as normal (you'll need to delete it yourself when you're done with it)

• have the results returned as a cursor, by specifying useCursor=True when you call aggregate.

Both of these options require mongodb 2.6: if you are still running mongodb 2.4 then this is just a fundamental limit of aggregations.

Answer 3

Use following snippet

db.patients.runCommand('aggregate', 
        {pipeline: [
    {"$project": {"birthday":1, "id":1}},
    {"$match": { "birthday":{"$gte":datetime.datetime(1987, 1, 1, 0, 0)} }}
], 
        allowDiskUse: true})

here allowDiskUse will help to find out exceed 16 MB data