Django Tastypie Advanced Filtering: How to do complex lookups with Q objects

Question

I have a basic Django model like:

class Business(models.Model):
    name = models.CharField(max_length=200, unique=True)
    email = models.EmailField()
    phone = models.CharField(max_length=40, blank=True, null=True)
    description = models.TextField(max_length=500)

I need to execute a complex query on the above model like:

qset = (
    Q(name__icontains=query) |
    Q(description__icontains=query) |
    Q(email__icontains=query)
    )
results = Business.objects.filter(qset).distinct()

I have tried the following using tastypie with no luck:

def build_filters(self, filters=None):
    if filters is None:
        filters = {}
    orm_filters = super(BusinessResource, self).build_filters(filters)

    if('query' in filters):
        query = filters['query']
        print query
        qset = (
                Q(name__icontains=query) |
                Q(description__icontains=query) |
                Q(email__icontains=query)
                )
        results = Business.objects.filter(qset).distinct()
        orm_filters = {'query__icontains': results}

    return orm_filters

and in class Meta for tastypie I have filtering set as:

filtering = {
        'name: ALL,
        'description': ALL,
        'email': ALL,
        'query': ['icontains',],
    }

Any ideas to how I can tackle this?

Thanks - Newton

Answer 1

You are on the right track. However, build_filters is supposed to transition resource lookup to an ORM lookup.

The default implementation splits the query keyword based on __ into key_bits, value pairs and then tries to find a mapping between the resource looked up and its ORM equivalent.

Your code is not supposed to apply the filter there only build it. Here is an improved and fixed version:

def build_filters(self, filters=None):
    if filters is None:
        filters = {}
    orm_filters = super(BusinessResource, self).build_filters(filters)

    if('query' in filters):
        query = filters['query']
        qset = (
                Q(name__icontains=query) |
                Q(description__icontains=query) |
                Q(email__icontains=query)
                )
        orm_filters.update({'custom': qset})

    return orm_filters

def apply_filters(self, request, applicable_filters):
    if 'custom' in applicable_filters:
        custom = applicable_filters.pop('custom')
    else:
        custom = None

    semi_filtered = super(BusinessResource, self).apply_filters(request, applicable_filters)

    return semi_filtered.filter(custom) if custom else semi_filtered

Because you are using Q objects, the standard apply_filters method is not smart enough to apply your custom filter key (since there is none), however you can quickly override it and add a special filter called "custom". In doing so your build_filters can find an appropriate filter, construct what it means and pass it as custom to apply_filters which will simply apply it directly rather than trying to unpack its value from a dictionary as an item.

Answer 2

I solved this problem like so:

Class MyResource(ModelResource):

  def __init__(self, *args, **kwargs):
    super(MyResource, self).__init__(*args, **kwargs)
    self.q_filters = []

  def build_filters(self, filters=None):
    orm_filters = super(MyResource, self).build_filters(filters)

    q_filter_needed_1 = []
    if "what_im_sending_from_client" in filters:
      if filters["what_im_sending_from_client"] == "my-constraint":
        q_filter_needed_1.append("something to filter")

    if q_filter_needed_1:
      a_new_q_object = Q()
      for item in q_filter_needed:
        a_new_q_object = a_new_q_object & Q(filtering_DB_field__icontains=item)
      self.q_filters.append(a_new_q_object)

  def apply_filters(self, request, applicable_filters):
    filtered = super(MyResource, self).apply_filters(request, applicable_filters)

    if self.q_filters:
      for qf in self.q_filters:
        filtered = filtered.filter(qf)
      self.q_filters = []

    return filtered

This method feels like a cleaner separation of concerns than the others that I've seen.

Answer 3

Taking the idea in astevanovic's answer and cleaning it up a bit, the following should work and is more succinct.

The main difference is that apply_filters is made more robust by using None as the key instead of custom (which could conflict with a column name).

def build_filters(self, filters=None):
    if filters is None:
        filters = {}
    orm_filters = super(BusinessResource, self).build_filters(filters)

    if 'query' in filters:
        query = filters['query']
        qset = (
                Q(name__icontains=query) |
                Q(description__icontains=query) |
                Q(email__icontains=query)
                )
        orm_filters.update({None: qset}) # None is used as the key to specify that these are non-keyword filters

    return orm_filters

def apply_filters(self, request, applicable_filters):
    return self.get_object_list(request).filter(*applicable_filters.pop(None, []), **applicable_filters)
    # Taking the non-keyword filters out of applicable_filters (if any) and applying them as positional arguments to filter()