Is there a way to augment django QuerySets with extra attributes?

Question

I\'m trying to add some extra attributes to the elements of a QuerySet so I can use the extra information in templates, instead of hitting the database multiple times. Let m

Answer 1

The error arises because qs.reverse() give rise to a new QuerySet instance, so you are not reversing the old one.

If you want to have a base QS on which to act you can do the following:

>>> augmented_books = Book.objects.extra(select={'price': 2))
>>> augmented_books[0].price
2
>>> augmented_books_rev = augmented_books.reverse()
>>> augmented_books_rev[0].price
2

Of course the select keyword can be much more complex, in fact it can be almost any any meaningful SQL snippet that could fit the [XXX] in

SELECT ..., [XXX] as price, ... FROM ... WHERE ... (etc)

EDIT

As pointed out in other responses, this solution may be inefficient.

If you are sure to get all the Book objects from the query, then you 'd better make one query, store it into a list and eventually reverse the resulting list.

If, on the other hand, you are getting the "head" and the "queue" of the table, making two queries is better, because you won't query all the "middle" useless objects.

Answer 2

This is an old question, but I'll add my solution because I needed it recently.

Ideally we could use some sort of proxy object for the QuerySet. Our proxy version would then make the changes during iteration.

It will be hard to cover all possible scenarios, QuerySet objects are a little complicated and are used in many different ways. But for the simple case of adding an attribute at the last minute because sending to a template (or generic view), the following might work:

class alter_items(object):

  def __init__(self, queryset, **kwargs):
    self.queryset = queryset
    self.kwargs = kwargs

  # This function is required by generic views, create another proxy
  def _clone(self):
    return alter_items(queryset._clone(), **self.kwargs)

  def __iter__(self):
    for obj in self.queryset:
      for key, val in self.kwargs.items():
        setattr(obj, key, val)
      yield obj

And then use this like so:

query = alter_items(Book.objects.all(), price=2)

Because it is not a true proxy, you may need to make further modifications depending on how it is used, but this is the rough approach. It would be nice if there were an easy way to make a proxy class in Python with new style classes. The external library wrapt might be useful if you want to go with a more complete implementation

Answer 3

If you iterate queryset before .reverse, then call the reverse and iterate the resulting queryset again then there will be 2 different SQL queries executed, .reverse() method won't reverse already fetched results, it will re-fetch the (possibly changed) results with an another SQL query. So what you're doing is not only fragile but also inefficient.

In order to avoid the second SQL query you can either reverse the queryset before iterating it or reverse a list with model instances in python using e.g. builtin 'reversed' function (see MattoTodd answer).

Answer 4

I would assume calling .reverse on a queryset s what is causing your issues. try this:

books = Book.objects.filter(author__id=1)
books_set = []
for book in books:
    book.price = 2
    books_set.append(book)

reverse = books_set.reverse()

Answer 5

Here is my version of alter_items proposed by Will Hardy.

Instead of a single value it allows different values of a custom attribute for each object: you can pass a mapping of values by object id.

It also automatically wraps the results of all methods that return QuerySet.

import types
from itertools import islice

from django.db.models import Model, QuerySet


class QuerySetWithCustomAttributes(object):
    def __init__(self, queryset, **custom_attrs):
        self.queryset = queryset
        self.custom_attrs = custom_attrs

    def __set_custom_attrs(self, obj):
        if isinstance(obj, Model):
            for key, val in self.custom_attrs.items():
                setattr(obj, key, val[obj.id])
        return obj

    def __iter__(self):
        for obj in self.queryset:
            yield self.__set_custom_attrs(obj)

    def __getitem__(self, ndx):
        if type(ndx) is slice:
            return self.__class__(self.queryset.__getitem__(ndx), **self.custom_attrs)
        else:
            return self.__set_custom_attrs(next(islice(self.queryset, ndx, ndx + 1)))

    def __len__(self):
        return len(self.queryset)

    def __apply(self, method):
        def apply(*args, **kwargs):
            result = method(*args, **kwargs)
            if isinstance(result, QuerySet):
                result = self.__class__(result, **self.custom_attrs)
            elif isinstance(result, Model):
                self.__set_custom_attrs(result)
            return result
        return apply

    def __getattr__(self, name):
        attr = getattr(self.queryset, name)
        if isinstance(attr, types.MethodType):
            return self.__apply(attr)
        else:
            return attr