MongoDB using an OR clause in mongoengine

Question

I'm using python's mongoengine to query MongoDB, and have loved it for the most part, but I'm having an issue with an advanced query.

Here's my model

class ContentItem(Document):
    account = ReferenceField(Account)
    creator = ReferenceField(User)
    public = BooleanField(default=False) 
    last_used = DateTimeField(default=datetime.now)

I would like to make a query for all ContentItem's that are of a particular account, and are either created by the logged in user or are public. Here's the query I wrote

query = ContentItem.objects.filter( (Q(account=account) & Q(public=True)) |  (Q(account=account) & Q(creator=logged_in_user)) ).order_by('-last_used')

or:

query = ContentItem.objects.filter( Q(account=account) & ( Q(public=True) |  Q(creator=logged_in_user) ) ).order_by('-last_used')

But these seem to be XOR where if either public, or the creator but not both. Is this expected?

Am I overlooking something? Should I do this directly with mongodb instead of mongoengine?

My current workaround is to make two different queries and combine the results, but as the # of Content Items gets larger the result is taking a long time to come back because I need to get all items before I can order them, thereby losing all the benefit of (django) paginated results.

Answer 1

The mongoengine documentation is apparently incorrect in this case. Instead of using the bitwise operators "&" and "|", you should use the standard operators "and" and "or".

So your first query becomes:

query = ContentItem.objects.filter( (Q(account=account) and Q(public=True)) or  (Q(account=account) and Q(creator=logged_in_user)) ).order_by('-last_used')

Answer 2

The correct way to do the query is to use bitwise operations | and & the way you wrote it in your question:

query = ContentItem.objects.filter( (Q(account=account) & Q(public=True)) |  (Q(account=account) & Q(creator=logged_in_user)) ).order_by('-last_used')

Note: using the standard Python boolean operators and and or will not work. This is explained in the MongoEngine documentation.

Answer 3

MongoEngine docs say otherwise. Please check this:

Answer 4

https://github.com/MongoEngine/mongoengine/blob/master/tests/queryset/transform.py

line 134

def test_raw_query_and_Q_objects(self):

    query = Foo.objects(__raw__={'$nor': [{'name': 'bar'}]})._query
    self.assertEqual(query, {'$nor': [{'name': 'bar'}]})

    q1 = {'$or': [{'a': 1}, {'b': 1}]}
    query = Foo.objects(Q(__raw__=q1) & Q(c=1))._query
    self.assertEqual(query, {'$or': [{'a': 1}, {'b': 1}], 'c': 1})

Answer 5

you are probably import the wrong Q

from mongoengine.queryset.visitor import Q as mongo_Q

from django.db.models import Q as normal_Q